System for and method of projecting augmentation imagery in a head-mounted display

ABSTRACT

A system for and method of projecting augmentation imagery in a head-mounted display is disclosed. A system for projecting light onto an eye includes a display to project light, a beam combiner, first and second optical systems between the display and the beam combiner along respective first and second optical paths. The first and second optical paths differ. The system also includes a switchable reflector that, in a reflective state, reflects light incident upon the reflector, and, in a non-reflective state, transmits light incident upon the reflector. The reflector is between the display and the first and second optical systems along the first and second optical paths and directs light along the first path, in the reflective state, or along the second path, in the non-reflective state, to reflect light from the beam combiner to the eye from different directions when in the different states.

RELATED APPLICATIONS

This application claims priority under 35 U.S.C. § 119(e) to U.S.Provisional Application No. 62/386,456, entitled “System and Method forAthletic Head Mounted Display of Augmentation Imagery”, filed Dec. 2,2015, and U.S. Provisional Application No. 62/372,270, entitled “Systemand Method for Aberration Correction of a Dynamic Region of InterestOver a Wide Field of View”, filed Aug. 8, 2016, each of which isincorporated by reference herein.

TECHNICAL FIELD

The technical field relates generally to projecting augmentation imageryin a head-mounted display, and in particular to systems and techniquesthat employ multiple beam launchers for image projection and systems andtechniques to correct image errors.

BACKGROUND

Our perception of the physical world is informed by our five senses:sight, hearing, taste, smell, and touch. As a consequence, if what wesense is altered our perception of reality is also altered. A primarysense is vision. Augmenting a user's vision of her surroundings withvirtual imagery powerfully adds to her perspective. This function isadvantageous in industry as well as for recreation. In order to providethis experience however, many new devices and subsystems must bedevised.

SUMMARY OF THE DISCLOSURE

The techniques described herein present improved systems and method forprojecting augmentation imagery in a head-mounted display.

In one aspect of the invention, a system for projecting light onto aneye includes a display configured to project light, a beam combiner, afirst optical system disposed between the display and the beam combineralong a first optical path, and a second optical system disposed betweenthe display and the beam combiner along a second optical path. Thesecond optical path is different from the first optical path. The systemalso includes a switchable reflector configured to selectively switchbetween a reflective state, in which light incident upon the switchablereflector is reflected by the switchable reflector, and a non-reflectivestate, in which light incident upon the switchable reflector istransmitted via the switchable reflector. The switchable reflector isdisposed between the display and the first and second optical systemsalong the first and second optical paths. The switchable reflectordirects the light along the first optical path to the beam combiner whenin the reflective state, such that the light is reflected off the beamcombiner and projected upon an eye from a first direction. Theswitchable reflector directs the light along the second optical path tothe beam combiner when in the non-reflective state, such that the lightis reflected off the beam combiner and projected upon the eye from asecond direction different from the first direction.

In an embodiment of the invention, the light is viewable by the eye in afirst field of view when the light is projected upon the eye from thefirst direction, and the light is viewable by the eye in a second fieldof view when the light is projected upon the eye from the seconddirection. Optionally, the first field of view is at least 30°.

In an embodiment of the invention, the first optical system isconfigured to project the light over the first field of view, the secondoptical system is configured to project the light over the second fieldof view, and the first field of view overlaps the second field of viewby at least 10°.

In an embodiment of the invention, the system also includes an eyetracking system configured to determine an orientation of the eyerelative to the beam combiner and a controller configured to switch theswitchable reflector between the reflective state and the non-reflectivestate based at least in part on the orientation of the eye.

In an embodiment of the invention, the switchable reflector has a clearaperture having a width of at least 2 mm.

In an embodiment of the invention, at least one of the first opticalsystem and the second optical system includes a foveated optical system.Optionally, the foveated optical system includes a liquid crystal wavefront corrector.

In an embodiment of the invention, the switchable reflector includes aliquid crystal mirror.

In an embodiment of the invention, at least one of the first opticalsystem and the second optical system is configured to collimate thelight, and the beam combiner is partially reflective.

In an embodiment of the invention, at least one of the first opticalsystem and the second optical system is configured to linearly polarizethe light.

In an embodiment of the invention, at least a portion of the beamcombiner is curved to collimate the light that is reflected to the eye.

In an embodiment of the invention, the first optical path is longer thanthe second optical path.

In an embodiment of the invention, a reflector is disposed between theswitchable reflector and the first optical system along the firstoptical path.

In an embodiment of the invention, the switchable reflector reflectssubstantially all light incident upon the switchable reflector in thereflective state.

In an embodiment of the invention, the display includes a firstsub-display that projects a first group of light rays having a firstresolution and a second sub-display that projects a second group oflight rays having a second resolution different than the firstresolution. The projected light includes the first group of light raysand the second group of light rays.

In another aspect of the invention, a method of projecting an image ontoan eye includes projecting light defining an image, via a display, ontoa switchable reflector and selectively switching the switchablereflector between a reflective state and a non-reflective state. Whenthe switchable reflector is in the reflective state, the light incidentupon the switchable reflector is reflected by the switchable reflectorand directed along a first optical path to a beam combiner, and thedirected light reflects off the beam combiner and is projected upon aneye from a first direction. When the switchable reflector is in thenon-reflective state, the light incident upon the switchable reflectoris transmitted via the switchable reflector and directed along a secondoptical path, different from the first optical path, to the beamcombiner, and the directed light reflects off the beam combiner and isprojected upon the eye from a second direction.

In an embodiment of the invention, the method also includes tracking anorientation of the eye relative to the beam combiner and the selectivelyswitching the switchable reflector is performed based at least in parton the orientation of the eye.

In an embodiment of the invention, when the switchable reflector is inthe non-reflective state, the light incident upon the switchablereflector is transmitted via a clear aperture defined in the switchablereflector, the clear aperture having a width of about 2 mm to about 10mm.

In an embodiment of the invention, projecting the light defining animage onto the switchable reflector includes projecting a foveatedimage.

In an embodiment of the invention, the switchable reflector includes aliquid crystal mirror.

In an embodiment of the invention, the first optical path is defined inpart by a first optical system configured to project the directed lightover a first field of view, and the second optical path is defined inpart by a second optical system configured to project the directed lightover a second field of view. The first field of view overlaps the secondfield of view by at least 10°. Optionally, at least one of the firstoptical system and the second optical system collimates the light thatdefines the image and the beam combiner is partially reflective.

In an embodiment of the invention, the display linearly polarizes thelight that defines the image.

In an embodiment of the invention, the beam combiner collimates thelight comprising the image that is reflected to the eye.

In an embodiment of the invention, the first optical path is longer thanthe second optical path.

In an embodiment of the invention, the method also includes reflectinglight from a reflector disposed between the switchable reflector and thefirst optical system along the first optical path.

In an embodiment of the invention, the switchable reflector reflectssubstantially all light incident upon the switchable reflector in thereflective state.

In an embodiment of the invention, at least one of the first opticalsystem and the second optical system include a liquid crystal wave frontcorrector.

In an embodiment of the invention, projecting the light includesprojecting a first group of light rays having a first resolution andprojecting a second group of light rays having a second resolutiondifferent for the first resolution. The projected light includes thefirst group of light rays and the second group of light rays.

In another aspect of the invention, an accommodating projection systemincludes a display configured to project light, a beam combiner disposedto at least partially reflect the projected light upon an eye, and aliquid crystal lens disposed along an optical path between the displayand the beam combiner configured to selectively focus the light. Theliquid crystal lens comprises a liquid crystal cell and an electrodelayer in electrical field communication with the liquid crystal cell.The electrode layer comprises a plurality of electrodes separated by aplurality of contour lines. The plurality of contour lines areconfigured according to a defocus wave front mode, and the defocus wavefront mode along at least one contour line has a value that is within30% of a constant wave front value.

In an embodiment of the invention, the light forms an image viewable bythe eye that is associated with a virtual distance from the eye.

In another embodiment of the invention, the system includes a liquidcrystal lens controller configured to update, at an accommodation updaterate, the liquid crystal lens based at least in part on the virtualdistance.

In another embodiment of the invention, the system also includes an eyetracking system configured to determine an orientation of the eye, andan eye tracking controller configured to determine the virtual distancebased at least in part upon the orientation of the eye.

In another embodiment of the invention, the system also includes adisplay controller configured to update the display at a selected framerate. Optionally, the accommodation update rate is equal to or greaterthan the frame rate. Further optionally, the accommodation update rateis about a whole number multiple of the frame rate.

In another embodiment of the invention, the liquid crystal lens has aclear aperture having a width of at least 2 mm.

In another embodiment of the invention, the display is configured tolinearly polarize the light. Optionally, the liquid crystal lens isconfigured to act upon linearly polarized light.

In another aspect of the invention, an accommodating projection methodincludes projecting light along an optical path incident on a liquidcrystal material disposed along the optical path and selectively varyingan index of refraction within the liquid crystal material in a patternthat is associated with a defocus wave front mode to selectively focusthe light. The method also includes at least partially reflecting thelight upon an eye.

In another embodiment of the invention, the light forms an imageviewable by the eye that is associated with a virtual distance from theeye.

In another embodiment of the invention, selectively varying the index ofrefraction of the liquid crystal material is based at least in part onthe virtual distance.

In another embodiment of the invention, selectively varying the index ofrefraction of the liquid crystal material occurs at an accommodationupdate rate.

In another embodiment of the invention, the method also includesdetermining an orientation of the eye, and determining the virtualdistance based at least in part upon the orientation of the eye.

In another embodiment of the invention, the method also includesprojecting the light at a frame rate. Optionally, the accommodationupdate rate is greater than or equal to the frame rate. Furtheroptionally, the accommodation update rate is about a whole numbermultiple of the frame rate.

In another embodiment of the invention, the method includes linearlypolarizing the light.

In another embodiment of the invention, the method includes collimatingthe light with an optic disposed along the optical path between thedisplay and the liquid crystal material.

In another aspect of the invention, an optical system for introducingwave front changes includes a liquid crystal cell and an electrode layerin electrical field communication with the liquid crystal cell. Theelectrode layer comprises a plurality of electrodes separated by aplurality of contour lines. The plurality of contour lines areconfigured according to an orthonormal wave front mode of an orthonormalbasis set. The orthonormal wave front mode along at least one contourline has a value that is within 30% of a constant wave front value. Theoptical system is configured to produce a wave front change associatedwith the orthonormal wave front mode that is at least 60% of an idealwave front change.

In another embodiment of the invention, the orthonormal basis setincludes a Zernike basis set.

In another embodiment of the invention, the wave front change has amaximum optical path difference of at least 3 waves.

In another embodiment of the invention, the wave front change is atleast 80% of an ideal wave front change.

In another embodiment of the invention, the electrode layer furthercomprises at least one transparent resistor configured to bridgeadjacent electrodes of the plurality.

In another embodiment of the invention, the system also includes afloating electrode layer between the electrode layer and the liquidcrystal cell. The floating electrode layer comprises a plurality offloating electrodes disposed between the contour lines of the electrodelayer and the liquid crystal cell.

In another embodiment of the invention, the orthonormal wave front modeis Zernike mode Noll index number 5.

In another embodiment of the invention, the system includes a controllerfor controlling an electrical potential of one or more electrodes.

In another embodiment of the invention, the liquid crystal cell has aclear aperture with a width of at least 2 mm.

In another embodiment of the invention, the liquid crystal cell isconfigured to act upon linearly polarized light.

In another aspect of the invention, a method for introducing wave frontchanges includes projecting light along an optical path incident on aliquid crystal material disposed along the optical path and selectivelyvarying an index of refraction within the liquid crystal material in apattern that is associated with an orthonormal wave front mode selectedfrom an orthonormal basis set to selectively change the light by atleast 60% of an ideal wave front change.

In another embodiment of the invention, the orthonormal basis setincludes a Zernike basis set.

In another embodiment of the invention, the wave front change has amaximum optical path difference of at least 3 waves.

In another embodiment of the invention, the wave front change is atleast 80% of an ideal wave front change.

In another embodiment of the invention, the orthonormal wave front modeis Zernike mode Noll index number 5.

In another embodiment of the invention, selectively varying the index ofrefraction within the liquid crystal material includes controlling anelectrical potential of at least one electrode in electrical fieldcommunication with the liquid crystal material.

In another embodiment of the invention, the method also includeslinearly polarizing the light.

In another embodiment of the invention, the pattern comprises aplurality of equal index of refraction regions delimited by a pluralityof contour lines. Optionally, the pattern along at least one contourline of the pattern associated with the orthonormal wave front mode hasa value that is within 30% of a constant wave front value.

In another embodiment of the invention, the pattern comprises aplurality of equal index of refraction regions delimited by a pluralityof equi-phase contour lines.

In another aspect of the invention, a jitter stabilization systemincludes a display configured to project light along an optical path toa location upon an eye, an augmentation jitter sensor that measures amovement of the eye relative to an environment, and an augmentationstabilization system disposed along the optical path between the displayand the eye that redirects the optical path based at least in part uponthe movement of the eye.

In another embodiment of the invention, the augmentation stabilizationsystem comprises a fast steering mirror.

In another embodiment of the invention, the fast steering mirror has aclear aperture having a width of at least 2 mm.

In another embodiment of the invention, the augmentation stabilizationsystem comprises a liquid crystal wedge. The liquid crystal wedgeincludes a liquid crystal cell and an electrode layer in electricalfield communication with the liquid crystal cell. The electrode layercomprises a plurality of electrodes separated by a plurality of contourlines. The plurality of contour lines are configured according to a wavefront mode of at least one of a tip wave front mode and a tilt wavefront mode.

In another embodiment of the invention, the liquid crystal wedge has aclear aperture having a width of at least 2 mm.

In another embodiment of the invention, the augmentation jitter sensorcomprises an accelerometer.

In another embodiment of the invention, the system also includes adisplay controller that updates the display at a frame rate, and theaugmentation stabilization system redirects the optical path at anaugmentation stabilization rate that greater than or equal to the framerate. Optionally, the augmentation jitter sensor measures the movementof the eye at a jitter sensor rate that greater than or equal to theframe rate.

In another embodiment of the invention, the system also includes anoptical jitter sensor that measures optical jitter, and the augmentationstabilization system redirects the optical path based at least in partupon optical jitter.

In another embodiment of the invention, the system also includes atleast one of a beam combiner disposed along the optical path between theaugmentation stabilization system and the eye and a waveguide disposedalong the optical path between the display and the eye.

In another aspect of the invention, a jitter stabilization methodincludes projecting light defining an image along an optical path to aneye. The image being viewable to the eye and appearing positioned at avirtual image location within a surroundings. The method also includesmeasuring a movement of the eye relative the surroundings and, based atleast in part upon the movement of the eye, redirecting the optical pathto cause the image to appear to remain positioned at the virtual imagelocation.

In another embodiment of the invention, redirecting the optical pathcomprises reflecting the light off a fast steering mirror.

In another embodiment of the invention, the fast steering mirror has aclear aperture having a width of at least 2 mm.

In another embodiment of the invention, redirecting the optical pathcomprises introducing a wave front change to the light of at least oneof a tip wave front change and a tilt wave front change by a liquidcrystal wedge disposed along the optical path.

In another embodiment of the invention, the liquid crystal wedge has aclear aperture having a width of at least 2 mm.

In another embodiment of the invention, measuring the movement of theeye comprises measuring a specific force exerted on a body that sharestranslational movements with the eye.

In another embodiment of the invention, the method also includesupdating the image at a frame rate, and redirecting the optical path isdone at an augmentation stabilization rate that is greater than or equalto the frame rate. Optionally, measuring the movement of the eye is doneat a jitter sensor rate that is greater than or equal to the frame rate.Further optionally, measuring optical jitter and redirecting the opticalpath is based at least in part upon the optical jitter.

In another embodiment of the invention, the projecting the lightdefining an image to the eye further comprising at least one ofpartially reflecting the light off of a beam combiner and transmittingthe light via a waveguide.

Additional aspects of the invention and additional features of thevarious embodiments of the invention are disclosed in more detail below.The phraseology and terminology employed herein are for the purpose ofdescription and should not be regarded as limiting. Moreover, any of theaspects and embodiments set forth above or otherwise herein may becombined with any of the other aspects and embodiments and remain withinthe scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows a diagram of a pupil forming architecture according tosome embodiments;

FIG. 1B shows a diagram of a non-pupil forming architecture according tosome embodiments;

FIG. 2A shows a head mounted display comprising a flat beam combineraccording to some embodiments;

FIG. 2B shows a head mounted display comprising a curved beam combineraccording to some embodiments;

FIG. 2C shows a head mounted display comprising a total internalreflection prism lens according to some embodiments;

FIG. 2D shows a head mounted display comprising a planar waveguideaccording to some embodiments;

FIG. 3 shows a human eye having a field of view;

FIG. 4 shows a graph depicting a distribution of rods and cones within anormal human eye;

FIG. 5 shows a block diagram of an athletic head mounted displaycomprising a number of subsystems according to some embodiments;

FIG. 6A shows a beam launching array according to some embodiments;

FIG. 6B shows an element of a beam launching array, or a “beam launchingcube” according to some embodiments;

FIG. 6C shows a curved beam launching array according to someembodiments;

FIG. 7 shows a schematic representation of a portion of an image fusionsystem, comprising: an HMD controller, an eye-tracking system, acompressor, and a beam launch array according to some embodiments;

FIG. 8 shows a graphical representation of a portion of an image fusionsystem, comprising: a beam launch array, and a beam combiner accordingto some embodiments;

FIG. 9 shows a field of view (FOV) of a user in horizontal and verticalplanes;

FIGS. 10A-10B illustrates an HMD in which two foveated optical systemsare employed to project augmented imagery into an eye of a useraccording to some embodiments;

FIG. 11 shows a reference embodiment of a wide field of view foveatedoptical system;

FIG. 12 shows a graph of Zernike Mode strength versus field angle for areference embodiment of a foveated optical system according to someembodiments;

FIG. 13A-13B illustrates Noll-ordered Zernike modes 5 and 6;

FIG. 14 shows a liquid crystal cell for an exemplary wave frontcorrector according to some embodiments;

FIGS. 15A-15B depict an etched electrode pattern for a wave frontcorrector that corrects Zernike mode 5 and Zernike mode 6 according tosome embodiments;

FIGS. 16A-16B depict an etched electrode pattern for a wave frontcorrector that corrects Zernike mode 2 and Zernike mode 3 according tosome embodiments;

FIG. 17 depicts an etched electrode pattern for a wave front correctorthat corrects Zernike mode 4 according to some embodiments;

FIGS. 18A-18B depict an etched electrode pattern for a wave frontcorrector that corrects Zernike mode 7 and Zernike mode 8 according tosome embodiments;

FIGS. 19A-19B depict an etched electrode pattern for a wave frontcorrector that corrects Zernike mode 9 and Zernike mode 10 according tosome embodiments;

FIG. 20 depicts an etched electrode pattern for a wave front correctorthat corrects Zernike mode 11 according to some embodiments;

FIG. 21 shows transparent resistors connecting adjacent transparentelectrodes according to some embodiments;

FIGS. 22A-22B illustrate Zernike modes at the pupil and image plane;

FIGS. 23A-23C contain corrected and uncorrected point source functionimages for a reference embodiment of a foveated optical system;

FIGS. 24A-24C contain corrected and uncorrected wave front images for areference embodiment of a foveated optical system;

FIGS. 25A-25B are plots showing resolution limit as a function of fieldangle, defining a region of interest for a reference embodiment of afoveated optical system;

FIG. 26 depicts a wide field of view having selectable regions ofinterest according to some embodiments;

FIG. 27 illustrates wave front changes to correct an astigmaticaberration according to some embodiments;

FIG. 28 illustrates wave front a stepped wave front compared to adesired non-stepped wave front according to some embodiments;

FIG. 29 illustrates wave front changes to correct a spherical aberrationaccording to some embodiments;

FIG. 30A shows an accommodating projection system according to someembodiments;

FIG. 30B shows a method of projected an image by scanning according tosome embodiments;

FIG. 31 shows a composition of a liquid crystal lens according to someembodiments;

FIG. 32A shows a scene over a number of focal depths according to someembodiments;

FIG. 32B shows and outlines a sharp field of view according to someembodiments;

FIG. 33 represents an embodiment of a head mounted display comprising aneye-tracking system according to some embodiments;

FIG. 34A represents a bright pupil measurement from a video-basedeye-tracking system according to some embodiments;

FIG. 34B represents a dark pupil measurement from a video-basedeye-tracking system according to some embodiments;

FIG. 35A shows a radial plot of a pupil position according to someembodiments;

FIG. 35B shows a radial plot of an estimated pupil position and anactual pupil position according to some embodiments;

FIG. 36A illustrates system movement yielding optical jitter accordingto some embodiments;

FIG. 36B illustrates system movement yielding augmentation jitteraccording to some embodiments;

FIG. 37 shows a representation of an augmentation stabilization controlsystem according to some embodiments;

FIG. 38 illustrates eye jitter in one dimension;

FIG. 39A shows a user's head in global and local coordinates accordingto some embodiments;

FIG. 39B illustrates head tracking accuracy according to someembodiments;

FIG. 39C illustrates head tracking precision according to someembodiments;

FIG. 40 shows an example layout of a commercially availablehead-tracking system;

FIG. 41 illustrates a cross-sectional views of an inner field of viewaperture stop and an outer field of view aperture stop according to someembodiments;

FIGS. 42A-D illustrate beam launching devices comprising raydensification means that project augmentation imagery within an innerand an outer field of view of a user according to some embodiments;

FIGS. 43A-C shows beam launching array rows comprising different meansof ray densification, or projecting an image at an inner and an outerfield of view of a user according to some embodiments;

FIG. 44 is a block diagram showing an HMD controller, comprising an HMDlocal processing according to some embodiments;

FIG. 45 illustrates exemplary rates for a number of HMD commandsaccording to some embodiments;

FIG. 46 depicts an exemplary A-HMD system according to some embodiments;

FIGS. 47A-B depicts a wave front corrector configured to introduce a tipgradient according to some embodiments;

FIG. 48 illustrates gradient anode patterns, optical path differencesresulting from gradient anode patterns, and a combined optical pathdifference according to some embodiments;

FIG. 49 shows a block diagram from an open and closed loop augmentationstabilization system according to some embodiments;

FIG. 50 illustrates an alternative embodiment of an augmentationstabilization system configured with multiple wave front correctors anda waveguide according to some embodiments;

FIG. 51 shows a schematic containing an inner and an outer projectionsystem for displaying imagery to a user according to some embodiments;

FIG. 52 represents Zernike modes;

FIG. 53 shows a schematic of a liquid crystal autorefractor according tosome embodiments;

FIG. 54 illustrates wave front control system comparison, comprising:jitter and focus correction, low-order control, and high-order controlsystems according to some embodiments.

FIG. 55 illustrates a side-by-side view, as displayed by a virtualreality (VR) HMD according to some embodiments;

FIG. 56 is an artistic rendering of stereo views produced by a VR HMDwith a foveated optical system according to some embodiments;

FIGS. 57A-57B depicts two different types of unmanned aircraftconfigured with foveated optical systems for imaging according to someembodiments;

FIG. 58 is an artistic rendering of an aerial photograph taken by acamera configured with a foveated optical system according to someembodiments; and

FIG. 59 depicts an alternative embodiment of a wide-angle foveated imagesystem configured with a rotatable wave front corrector according tosome embodiments.

FIGS. 60-93 show full sized corrected and uncorrected point sourcefunction images corresponding to those of FIGS. 23A-23C.

FIGS. 94-127 show full sized corrected and uncorrected wave front imagescorresponding to those of FIGS. 24A-24C.

DETAILED DESCRIPTION

Augmented Reality (AR) is a technique used to enhance a user'sperception of the world by augmenting their normal sensory input withcomputer-generated imagery. Taken to the extreme, if a user's perceptionis completely replaced by a simulation, one can create a pure “virtualreality” (VR). Mixed Reality (MR) combines these techniques to merge auser's perception of physical reality with a VR simulation thatco-exists with the physical world in space and time.

The merging of physical and virtual realities creates a new experienceof reality where a user feels like they are “inside” the computersimulation. With MR, a user senses both virtual and physical objects inthe same space, at the same time, and as a consequence both types ofobjects have the potential to be perceived as “real”.

Mixed reality gaming is the fusion of a physical sport with a virtualreality simulation that enables a new kind of experience. A novel mixedreality gaming experience is expected to provide participants with aform of recreation that is more immersive and sophisticated than currentgaming. A mixed reality gaming experience may be imagined by the reader,as being similar to playing your favorite video game in a 3D IMAXtheater. Except: you are not sitting down, your opponents may be present(physical) or remote (virtual), and the environment you are playing incomprises the physical world.

Computer games have become increasingly popular in recent years.According to a recent report, the computer gaming industry may soon growto a size of $100 Billion. Using the sheer size of the gaming industryas grounds, it is fair to infer that many find pleasure in gaming. Themixed reality gaming experience, as proposed within, is far superior tothe experience offered by current computer games. It is thereforeexpected, that mixed reality gaming has the potential to offer greatnumbers of people untold hours of pleasure. Currently, the benefits ofmixed reality gaming have yet to be realized. This is in large part dueto: limitations of optical systems present in current augmented realitytechnologies. A technology that's current state precludes therealization of mixed reality gaming is head mounted displays.

A head mounted display should provide the augmentation imagery in a waythat is comfortable to most users. For decades it has been understood bymany stereoscopic cinema experts, that affected individuals find 3Dmovies, or stereoscopic head mounted displays, uncomfortable. It iswidely believed that this discomfort is often, because theseindividuals' stereoscopic perception of depth is coupled with thefocusing of their eyes. In 3D cinema, an image having positive parallaxis perceived by a viewer to be closer to them than the screen. However,light rays that comprise the image are reflected from the screen.Therefore in order to see the image clearly and in 3D, a viewer focusestheir eyes as if the image is located at the screen, while perceivingthe image at a different depth than the screen. Some affectedindividuals have difficulties focusing properly on the image, andmaintaining a stereoscopic perception of depth. The perceivedincongruence between focal depth and stereoscopic depth thus causesthese individuals discomfort. An athletic head mounted display providesan image having stereoscopic depth, or parallax, such that theaugmentation imagery has a perceivable depth. An A-HMD therefore,displays the augmentation image with a focal depth that is congruentwith the stereoscopic depth of the image.

An athletic head mounted display should also provide high resolutionaugmentation imagery to the user. Augmentation imagery that has poorresolution will jump out as false, when collocated with physical objectsin sight nearby. Therefore, augmentation imagery should be delivered tothe user in lifelike or near-lifelike resolutions.

A head mounted display (HMD) for mixed reality gaming, or an athletichead mounted display (A-HMD), as shown and described herein, provides agame participant with an unbroken coherent perception of theaugmentation imagery. To realize this, augmentation imagery is displayedto a participant over a full field of view and over a full range of eyemovement. in other words, the augmentation imagery should not appearonly when the participant's eyes are straight forward, or when theparticipant adjusts her eyes so that the augmentation imagery is in hercentral vision.

The augmentation imagery should appear to the participant at all timesto be located in the physical world. That is to say, the augmentationimagery should appear to stay in the same location, relative thephysical environment, as the participant moves her head. In the case ofan athletic head mounted display, participants are expected to bephysically active. Therefore, An A-HMD should update a location of theaugmentation imagery on the participant's eye faster than theparticipant can perceive as she moves her head.

Information relevant to attempts to address these problems can be foundin U.S. application Ser. No. 61/909,774, U.S. Patent Application No.2014/0232651 and U.S. Pat. No. 8,780,014. However, each one of thesereferences suffer from one or more of the following disadvantages: Thesystem taught by the reference is unable to sense a movement of a user'seye and reposition projected imagery, based upon the eye movement. Thesystem taught by the reference does not selectively project imagery intodistinct ranges of field of view of the user, such as central vision,near-, mid-, and far-periphery. The system as taught by the referencetherefore is unable to project imagery at the central vision of a user,which is of a higher resolution than imagery being projected in aperiphery vision of a user. The system taught by the reference does notallow augmentation imagery to be corrected and steered within limits ofperceivable resolution of the user. The system as taught in thereference cannot be optically calibrated based upon the uniquecharacteristics of a user's eyes. And, the system taught by thereference is unable to correct jitter of projected imagery that resultsfrom fast movements of a user's head.

For the foregoing reasons, there exists a need for a head mounteddisplay that can allow people to enjoy an immersive and sophisticatedmixed reality gaming experience.

Unlike virtual reality gaming, mixed reality gaming requires display ofaugmentation imagery amongst physical surroundings, which are alsoperceptible to the user. See through head mounted displays (HMDs) allowfor the display of virtual content, while largely not obstructing theuser's view of her surroundings.

There are primarily two types of optical system architectures for seethrough HMDs: pupil forming and non-pupil forming architectures. Forcomparison, both architectures are illustrated in FIG. 1. In pupilforming architectures, there are one or more intermediate pupil planes,110, between an image source, 112, and a final pupil plane, 114, being apupil of a user's eye. In a non-pupil forming architecture, there isonly the one pupil plane, 114, and no intermediate pupil planes. FIG. 1Aillustrates a pupil forming HMD architecture having an intermediateimage, 110. FIG. 1B shows a non-pupil forming architecture with aviewing eye box, 116.

Pupil forming architectures allow flexibility in the optical designlayout (e.g. head wrapped optical train) and optical systemfunctionality (e.g. eye tracking capability). A non-pupil formingoptical architecture tends to be much more compact with a viewing eyebox, 116, that tends to be wider in depth and narrower in lateraldirections.

Identifying how the transmitted and projected beams are combined furtherrefines the optical system architectures of an HMD. There is a widerange of see-through HMD architectures that take advantage of severaloptical fields such as: traditional optics, Fresnel optics,micro-optics, diffractive optics, and volume holograms. All thearchitectures provide some capability while also yielding somedrawbacks. Typical HMD beam combining methods are drawn in FIG. 2.

The simplest see-through HMD is illustrated in FIG. 2A, and makes use ofa conventional flat combiner, 210. An embodiment incorporating the flatcombiner, 210, uses a simple 50/50 beam splitter tilted at 45 degrees tocombine the beams. Another interesting design employs curved combiners,212, which is illustrated in FIG. 2B. The curved combiner, 212,comprises one or more curved surfaces that collimate the projected beam,214, and project it onto the user's eye, 216. Curved combiners may bemade semi-transparent by the proper application of thin film coatings.Numerous companies have attempted to make use of a total internalreflection (TIR) prism lens, 218, as a beam combiner. The TIR prism lensis illustrated in FIG. 2C. A beam combining system that implements a TIRprism lens consists of a micro-display mounted on top of the combinerprism. A second corrector element, not shown, is used to restoresee-through functionality. A current favored method for HMD beamcombining comprises a planar waveguide, 220. An exemplary configurationof such a system is shown in FIG. 2D. The planar waveguide, 220, havingan input coupler, 222, and an output coupler, 224, may be input, and/oroutput coupled with diffraction gratings, holograms, or micro-mirrorarrays. Other HMD beam combiner methods include: cascaded extractors,dual contact lens/retro-diffuser, and tapered light guides.

An athletic HMD (A-HMD) is an HMD that enables athletic mixed realitygaming. An athletic HMD enables a user to sense simulated objects as ifthey were real, as the user and her eye's move uninhibited in athleticaction. This demands strict performance requirements not yet met bycurrent HMD systems. Ideally, an A-HMD would allow for sensed simulatedobjects to be perceivable over a full field of view and an entire rangeof eye movement.

As depicted in FIG. 3, a human eye, 302, has an instantaneous field ofview (iFOV), 304, that is greater than 120° per eye. However, the eyedoes not maintain the same vision quality over the entire FOV. Thevarying vision quality is because of the structure of the eye and alight-sensitive layer at the back of the eye called a retina, 306. Lightentering a pupil, 308, projects upon the retina, which covers roughly65% of the eye's interior surface. The retina, 306, is comprised of rodsand cones. Rods and cones are photosensitive cells in the retina thatconvert incident photons into electrical signals that are carried to thebrain via the optic nerve, 310. Cones are color sensitive cellsapproximately 6 microns in diameter that perform well in bright lightingbut are insensitive in low-light conditions. Rods are highlylight-sensitive cells approximately 2 microns in diameter that performwell in low-lighting conditions. Rods are not able to distinguish color;and rods do not form as sharp an image as the cones do. The majority ofthe cones are located at the center of the retina on a small bump calleda fovea, 312.

The distribution of rods and cones on the retina, combined with otheranatomical considerations in the eye, results in their being regions ofthe field of view with different vision qualities: central vision, 314,and peripheral vision, 316.

Central vision, 314, may be defined by the boundaries of the macula,which is a region of the retina defined as having at least two layers ofganglia. The macula has a diameter of 5.5 mm or 18° FOV. The definitionof central vision may also be defined in terms of visual acuity. In thiscase, central vision is referred to as foveal vision and may be definedas the region of the FOV in which visual acuity is 20/20. Thiscorresponds to roughly the central 2° FOV.

Peripheral vision, 316, may be defined as the region of the FOV outsidethe central vision region. Peripheral vision may be divided into threeregions: near, mid, and far. The dividing line between near and midperipheral vision may be defined according to visual acuity, whichdeclines by about 50% every 2.5° from the center of the FOV up to 30°where it falls off more sharply. This sharp decline in visual acuitydefines the boundary between near and mid peripheral vision. Inaddition, color perception is strong at 20° but weak at 40° and the 30°boundary between near and mid peripheral vision is also taken as theboundary between adequate and poor color perception. This boundary alsonaturally occurs for dark-adapted vision as well, where the 30° boundarycorresponds to the edge of good night vision, primarily resulting fromthe distribution of rods in the retina. Far peripheral vision may bedefined as the region of vision that lies outside stereoscopic vision,which is defined as the region of the FOV that overlaps for each eye.This overlap occurs for the inner 60° of FOV and allows people toperceive depth stereoscopically.

There are approximately 100 million rods and 5 million cones spreadacross the retina. As alluded to above, the rods, 402, and cones, 404,are not distributed uniformly over the retina. The density of rods, 406,and the density of cones, 408, over a retina is graphed in FIG. 4. Apeculiarity of the retina is the presences of an optic disk, or opticnerve head, 410, where there are no rods or cones resulting in a seldomperceived blind spot. The fovea contains the highest density of cones,408, and provides the eye's sharpest vision and the majority of theeye's color vision.

An athletic HMD (A-HMD) may be defined as an HMD that enables athleticmixed reality gaming. To do so, an HMD should enable a user to sensesimulated objects as if they were real. In order to achieve this, theHMD should provide greater functionality than achieved by prior methods.Ideal specifications of four functions of an A-HMD are defined in termsof human perfectibility, and are not necessarily met by all embodiments:An A-HMD should provide stereoscopic display of augmentation imagery,which enables normal eye accommodation for a user. Normal eyeaccommodation of stereoscopic imagery prevents discomforting experiencesmany affected individuals experience when viewing 3D content. Someembodiments of an A-HMD include an accommodating projection system thatdisplays stereoscopic augmentation imagery with more than four focaldepths. An A-HMD should update augmentation imagery at a rate that isimperceptible to the human eye. Some embodiments of an A-HMD comprise aframe rate that updates the augmentation imagery content at a rategreater than 10 Hz, such as 30 Hz. An A-HMD should display augmentationimagery, such that it appears to be placed stably amongst thesurroundings of the real word. Some embodiments of an A-HMD correctaugmentation imagery instabilities, to within the resolution limit of ahuman eye, or within +/−10 microns on a user's retina. An A-HMD shouldprovide the greatest possible field of view, resulting in the mostimmersive mixed reality experience for the user. Some embodiments of anA-HMD allow a user to see a full +/−60° FOV horizontally and +50°/−70°FOV vertically

A head mounted display able to meet the functional specifications asoutlined above, may comprise any of the following subsystems: Ahead-tracking system, an eye-tracking system, an accommodatingprojection system, an augmentation stabilization system, a raydensification system, and a head mounted display controller.

A block diagram of an HMD system, comprising a number of subsystems isshown in FIG. 5, having data flow, 501, from left to right. In respectto FIG. 5: An HMD Controller, 502, receives externally generated scenerender data, 504, initiating the process. The HMD Controller, 502, alsoreceives internally generated head-tracking, or jitter sensing data andeye-tracking data from a jitter sensor system, 506, and an eye-trackingsystem respectively, 508. The HMD Controller, 502, controls a pluralityof subsystems in order to generate augmentation imagery, 509, on theretina of a user's eye. The plurality of subsystems may comprise one ormore of the following: An accommodating projection system, 510, anaugmentation stabilization control system, 512, an image fusion system,514, and a ray densification system, 516.

Image Fusion System

An image fusion system performs the critical task of mixing the real andaugmented imagery, so that a user perceives the scene as the fusion ofthe two. Referring now to FIG. 6, in some embodiments, an A-HMD system,comprises a beam launcher array (BLA), 602. Versions of image fusionsystems comprising a BLA allow a user to have access to a significantlylarge FOV, for example greater than 25%, 50%, 70% of an iFOV of a user.Said embodiments also, allow the user to turn her eyes and see asignificantly large FOV in most gaze directions. The BLA is anoutput-coupling device that allows light, 604, output by a projector tostay locked on the pupil of a user's eye. The beam launcher array (BLA),as depicted graphically in FIG. 6A, provides image fusion over large FOVand many eye orientations.

The BLA comprises: an array of beam launching cubes (BLCs), 606. TheBLCs are so named, because they may be shaped as cubes. Embodiments ofBLCs exist that are not shaped like cubes. In an embodiment shown inFIG. 6B, the BLCs are constructed by placing a switchable reflector,608, along one diagonal face of a substrate, 610. A first adjustablegradient optic, 612, is placed along an input surface, 614, of thesubstrate, 610, and a second adjustable gradient optic, not shown, maybe placed at a first output surface, 616, of the substrate, 610. Theswitchable reflector, 608, is selectively switched from transparent toreflective. When the switchable reflector is in a reflective (ON) state,rays entering the input surface are reflected out a second outputsurface, 618. When the switchable reflector, 608, is in a non-reflective(OFF) state, the light, 604, passes out the first output surface, 616.In some embodiments, the switchable reflector, 608, comprises a liquidcrystal mirror (LCM); and the LCM may be switched from a reflective to anon-reflective state at a rate of about 120 Hz. In some embodiments thevariable gradient optic, comprises a liquid crystal wedge. In someversions, the BLA's are sized having a clear aperture, or aperture stopthat is large enough to selectively reflect all of the light associatedwith an image. For example, A BLA may accommodate a similarly sizedimage width, by having a clear aperture of 1, 3, or 5 mm.

In some embodiments, A liquid crystal wedge (LCW), 612, placed at aninput face, 614, of the beam launcher is used to steer an input beam,604, vertically or horizontally. The LCW, 612, adds a tip or tilt phasegradient to a pupil, which shifts the vertical or horizontal position ofthe augmentation imagery. The LCW works in a similar fashion to a liquidcrystal lens (LCL), 620; a potential is applied across the device, andthe index of refraction of the liquid crystals changes. Unlike anannular, ring pattern electrode, on the LCL, 620, the LCW, 612, has anelectrode pattern comprising: a series of strips through which anon-uniform potential is applied across the whole device. A phasegradient (tip/tilt) may be introduced by varying the potential in alinear ramp, causing the input beam, 604, to deflect at an angle equalto the induced wedge angle.

In some embodiments, a liquid crystal lens (LCL), 620, as described infurther detail below, is placed at the second output surface, 618, ofthe beam launcher cube to expand the output beam, 604. In combinationwith a curvature of a curved beam combiner, as described above, thefunction of the LCL, 620, provides the ability to increase the size ofthe output beam, 622, to match the diameter of a user's pupil, 308.Light launched from the BLA reflects off the partially mirrored surfaceof the beam combiner into the user's eye. In some versions, thecurvature of the beam combiner is matched to the curvature of a curvedBLA, 624, in order to collimate the projected beam entering the user'seye. A wide FOV may be achieved by the curved BLA, 624, as shown in FIG.6C.

In some embodiments a beam launching array is a two dimensional array,having a first column of beam launching cubes into which light is inputcoupled. The first column is substantially disposed along a first axis(Y). The light is selectively reflected and output coupled from thefirst column and into any one of a multitude of rows of beam launchingcubes. Each row of beam launching cubes comprises a multitude of beamlaunching cubes, and is disposed along a second axis (X). In someembodiments the first (Y) and second (X) axes are generally orthogonalto one-another. The light is finally selectively reflected and outputcoupled from a beam launching cube within a row, and ultimately directedtoward the eye of a user. Thus, light may be selectively launched fromthe two dimensional beam launching array according to a coordinate (X′,Y′).

An embodiment of an image fusion system, up to and including the beamlauncher array (BLA), 702, is depicted schematically in FIG. 7. Theimage fusion system is depicted graphically from the BLA, 702, to aneye. Referring to FIG. 7, light, 704, from an entrance pupil isdecreased in size by a compressor, 706, such that the diameter of theprojected beam matches an entrance aperture, not shown, of the BLA, 702.In the version an image fusion system shown, the compressor, 706,comprises a first compressor lens, 708, and a second compressor lens,710. Additionally, the compressor may comprise: curved reflectors,diffractive gratings or other optical elements. The projected light,704, then passes into the BLA, 702, and is routed to a correct beamlaunching cube, 712, based upon pupil orientation measured by aneye-tracking system, 714. An HMD controller, 716, controls theperformance of the beam launch array, 702, in response to feedback froman eye tracking system, 714.

The projected beam, 802, is launched from the BLA, 804, onto a beamcombiner, 806, as illustrated in FIG. 8. In some embodiments, the beamcombiner, 806, comprises a glass surface that is partially mirrored.External light sources, 808, shine through the beam combiner into theuser's eye, 810, and projected augmentation imagery, 812, is reflectedfrom the partially mirrored surface into the user's eye, 810, asdepicted in FIG. 8. This effectively combines the real, 808, andaugmentation imagery, 812, together into one scene, as viewed by theuser's eye.

Field of View (FOV) Specifications

A typical person has a large field of view (FOV), as depicted inhorizontal and vertical planes in FIG. 9. A horizontal visual limit,902, for each eye is roughly 62° off center. A region of binocularvision, 904, typically is present in between the horizontal visuallimits, 902. Symbol recognition, 906, is typically only possible within5°-30° of the standard line of sight, 908. The standard line of sight,908, is generally 15° higher than the seated line of sight, 910. Optimumvertical eye rotation, 912, is between an upper maximum eye rotation,914, of about 25°, and a lower maximum eye rotation, 916, of about 30°.Vertical limits of vision are about 50° upper limit to visual field,918, and 70° lower limit to visual field, 920. Referring back to FIG. 3,the instantaneous FOV (iFOV), 304, is the FOV that an eye is able toimage at a given instance. The level of mixed reality immersion isproportional to the amount of the iFOV that may be augmented with ARcontent, which we shall refer to as the AR-FOV. An AR device is said tobe immersive if the AR-FOV is a significant fraction of the eye's iFOV.

Total accessible FOV is an amount of total FOV that is accessible by auser's eyes when the eyes are rotated. Ideal A-HMD FOV specifications,which are not necessarily achieved in all embodiments, are: An AR-FOVthat provides augmentation imagery over at least 70% of the totalinstantaneous field of view of an average human eye. And, an accessibleFOV that provides augmentation imagery over at least 70% of an averagehuman eye's total accessible FOV. According to some embodiments, theAR-FOV is about 50% allowing for augmentation imagery to be projected,only within the limits of stereoscopic vision. In another version, theAR-FOV is further reduced to about 25% the iFOV of the user, andaugmentation imagery is only projected within the user's central visionand near-to-mid peripheral vision. In another embodiment, the AR-FOV isstill further reduced, such that it includes only the central vision andthe near-peripheral vision. Like the AR-FOV, the accessible FOV of thesystem is also ideally as large as possible, although in someembodiments it is as low as 50%. In other embodiments the accessible FOVis hardly constraining to a user and represents greater than 70%, 90%,or 97% of the iFOV.

As described above, the human eye, 302, has a relatively largeperipheral vision, 316 (referring back to FIG. 3). It is thereforenecessary for a near-to-eye projection system, such as a head mounteddisplay (HMD), to display over a wide field of view, greater than 30°and, according to some embodiments greater than 60°. A 2D beam launcherarray, 1002 (depicted in FIGS. 10A-B) may be employed to displayaugmentation imagery over a large accessible-FOV and provide a largeAR-FOV and an immersive mixed reality experience. According to someembodiments, the HMD comprises multiple optical systems that areconfigured to provide a wide total accessible FOV greater than 60°, andin some embodiments greater than 120°.

An X-Z cross section view of an embodiment of an HMD incorporatingmultiple optical systems is shown in FIG. 10A. A Y-Z cross section viewof the same HMD embodiment is shown in FIG. 10B. The HMD depicted inFIGS. 10A-B includes a beam combiner, 1004. The beam combiner reflectsprojected light being reimaged from an image plane, 1006, by the 2D beamlauncher array, 1002, toward an eye of a user, 1008. In some embodimentsthe beam combiner partially transmits light from the surroundings, 1010,allowing some of it to reach the user's eye, 1008. Examples of beamcombiners include: curved and flat 50/50 beam splitters. As a user's eyemoves, multiple optical systems, 1012 and 1014, selectively providevirtual imagery over the total accessible FOV of the user, see FIG. 10A.In such situations, one or more switchable reflectors, or liquid crystal(LC) mirrors, 1016, may be used to direct projected light toward anindividual optical system. The switchable reflector can be said to havea clear aperture, 1017, which constrains the area of light transmittedor reflected by the switchable reflector. In some embodiments, theminimum overall clear aperture of the 2D beam launching array isconstrained by the switchable reflector's clear aperture, 1017. Alsoshown in FIG. 10A is a reimaging lens, 1018, and a reflector, 1020. Thereimaging lens is used to correct for differences in path length betweenthe two optical systems. The reflector further directs the projectedlight through the second optical system, 1014.

Referring now to the function of the embodiment shown in FIGS. 10A-10B,an image generating source (OLED display, projector, LCOS device, etc.)projects light forming an image at an image plane, 1006. In someembodiments, the projected light is linearly polarized, and thereforeefficiently acted upon by liquid crystal devices. The light propagatesfrom the image plane and passes through (or is reflected by) aswitchable reflector, 1016. When the switchable reflector is in anon-reflective state; light from the image source passes through a firstbeam launcher, 1012. When the switchable reflector is in a reflectivestate; light from the image source passes through the second beamlauncher, 1014. In either case, the light, 1021, passing through thebeam launchers is directed toward a beam combiner, 1004. The beamcombiner finally reflects the light from the source into a user's eye,1008. The system is typically symmetrical for both eyes of the user,such that each eye has virtual imagery displayed to it. In someembodiments, the light passing through the beam launchers, or opticalsystems, 1012 and 1014, is collimated by one or more of the opticalsystems (such as is described below in reference to FIGS. 42A-42D) andthe collimated light is reflected off a partially reflective beamcombiner, 1004, and into the user's eye. In another embodiment, thelight from the beam launchers is not collimated, and a curved beamcombiner is incorporated to collimate the light that is reflected to theeye.

Referring again to FIG. 10A, an HMD is partially shown, in which twooptical systems are incorporated per eye. A first optical system (OS),1012, images over a first wide angle, 1022. A second OS, 1014, imagesover a second wide angle, 1024. In some embodiments, the wide angle isgreater than: 45°, 60°, or 80°. It can be seen in FIG. 10A that thefirst OS, 1012, and the second OS, 1014, are positioned such that theyoverlap at an overlap angle, 1026. In some embodiments, the overlap,1026, is negligible. In other embodiments, the overlap angle, 1026, isgreater than: 60°, 45°, 30°, or 10°. In some embodiments, adjacent OSsare configured such that there is an eye overlap angle, 1028, between athird OS associated with a second eye, not shown, and the first OS,1012. In some embodiments, the eye overlap, 1028, is negligible. Inother embodiments, the eye overlap angle, 1028, is greater than: 10°,30°, 45°, or 60°.

Multiple optical systems, or beam launchers, may be incorporated todisplay an image to an eye. Parameters related to beam launchers areoutlined in Table 1, below:

TABLE 1 Min. Nom. Max. Beam Launcher - Clear 0.5 mm 4-10 mm 50 mmAperture (mm) Beam Launcher - FOV 2% 40%-60% 100% (% iFOV*) BeamLauncher - FOV 2° 50°-70° 120° (°) No. Beam Launchers per 1 2-5 100 Eye(—) Multiple Beam Launchers - 30%  50%-90% 100% Accessible-FOV (% TotalAccessible-FOV) Multiple Beam Launchers - 60° 100°-160° 180°Accessible-FOV (°) Operable Wavelengths: Broadband within 400-700 nmrange, and Monochromatic, and IR 700-1500 nm. Type of Optical System:Pupil forming, and Non-pupil forming *Assume an instantaneous Field ofView (iFOV) of 120° for a nominal human eye. ** Assume 180° of totalaccessible Field of View (TA-FOV) for a nominally rotatable human eye.

In some embodiments the optical systems (OS) used to direct the light totoward the eye are foveated optical systems (FOS). A foveated opticalsystem may be described as an optical system that provides varyingresolutions over a field of view, much like the human eye which imagesat a high resolution only at the fovea. Therefore, an embodiment of aFOS has a region of interest (ROI) that is imaged at a fovealresolution, outside of which the image has a resolution lower than thefoveal resolution.

Introduction to Foveated Optical System

Because angular resolution is inversely proportional to wavelength oflight divided by aperture diameter, larger aperture sizes result in highresolution imaging. An eye's aperture, or pupil, can vary in diameterfrom approximately 1.5 mm in bright light, to approximately 8 mm in dimlight. Therefore, generally speaking, for an optical system to projectimages of imperceptibly high resolution to the eye, the optical systemmust have an aperture size of at least 5 mm for average viewingconditions. It is advantageous for an HMD optical system to project widefield of view images at resolutions, which are imperceptible to thesurrounding images. Therefore, an HMD optical system with an aperturesize at least as large as a human pupil, and which can pass enough lightfor high resolution imaging is required. A wide angle foveated opticalsystem may be used to display augmentation imagery over a large field ofview, and with high-resolution. Wide angle foveated optical systems aredescribed in U.S. application Ser. No. 14/020,223 (Li et al.); Ser. No.14/726,429 (Welch et al.); and U.S. Pat. No. 8,018,814 (Ogasawara elal.), incorporated by reference herein. However, each one of thesereferences suffer from one or more of the following disadvantages: anon-dynamic region of aberration correction, small optical pathdifferences, and fill factor effects. A non-dynamic region of interestmeans that a region imaged at high resolution is static, or in one placein the field of view. For a mixed reality application, the static regionof interest will constrain the user's eye movements and hobbleimmersivity. Small optical path differences limit the aberrationscorrected by the optical system, thus reducing the quality of thedisplayed imagery and mixed reality (MR) experience. Fill factor effectsare the result of the use of spatial light modulators, and reduces thetransmission efficiency of the optical system.

A doctoral thesis titled: ANALYSIS AND DESIGN OF WIDE-ANGLE FOVEATEDOPTICAL SYSTEMS by George Caratu describes design considerations forwide-angle foveated optical systems. The designs outlined by Caratusuffer from the following disadvantageous: fill factor effects, reducedtransmission, limited to monochromatic applications, and small phasestroke. For example, Caratu describes systems that are limited to aphase stroke of about 1λ, and describes that phase stroke will be smallenough in compact wide-angle systems, such that aberration correctionwill require modulo-λ techniques. Modulo-λ corrections however severelylimit the wavelength that can be corrected, as diffraction efficiencydrops quickly away from the center wavelength.

For the foregoing reasons, improvements in technology related towide-angle foveated imaging is needed, in order for a truly immersivemixed reality experience.

Foveated Optical System: Parameters Table

Parameters related to foveated optical systems are outlined in Table 2:

TABLE 2 Minimum Reference Maximum No. Optical Elements (—) 1 3 100 Note:A fewer number of elements is typically preferred. Clear ApertureDiameter (mm) 0.5 mm  6 mm  50 mm Note: an aperture size at least aslarge as a human pupil is generally preferred. No. Wave Front Correctors(—) 1 1  50 Field of View (°) 20° 60° 180°  Region of Interest (°)  2° 8° 90° System Thickness (mm) 0.1 mm ~25 mm >100 mm Type of OpticalSystem pupil forming non-pupil forming Application Display Systems:Augmented Reality HMD, Virtual Reality HMD, etc. Imaging Systems:Surveillance, Autonomous vehicle camera systems. etc.

In some embodiments, such as a reference embodiment shown in FIG. 11, afoveated optical system, 1101, comprises three optics: two even asphericlenses, 1102 and 1104, in a fisheye projector configuration, with aliquid crystal (LC) wave front corrector, 1106, positioned between them.This configuration images light over a wide field of view. For example,the reference embodiment provides a field of view of 60°. Generally,increased field of view can result in greater aberrations. In thereference embodiment, the two aspheric optics, 1102 and 1104, areconfigured to correct azimuthally symmetric aberrations, but do notcorrect azimuthally asymmetric aberrations. Depending upon the design ofthe foveated optical system, 1101, different aberration sources willcontribute more or less. Specifications for the aspheric optics, 1102and 1104, of the reference embodiment are included below in Table 3:

TABLE 3 Lens No. 1 Lens No. 2 Numerical 42A 42B Indicator TypeEVENASPHERE EVENASPHERE Thickness 2.66 mm 6.00 mm Material SF57_MOLDSF57_MOLD Front Back Front Back Semi-Diameter 3.71 mm 2.39 mm 4.15 mm5.89 mm Conic 0 0 0 0 2nd Order 0 0 0 0 Term 4rd Order Term 6.88E−056.83E−04 −1.81E−04 −4.33E−05 6th Order Term −1.34E−06 −1.03E−06−3.26E−06 −7.46E−07 8th Order Term 2.72E−08 1.48E−06 1.74E−07 −2.76E−0910th Order −6.84E−10 2.98E−09 −5.98E−10 −3.06E−10 Term

Many liquid crystal materials act most efficiently on linearly polarizedlight. Some embodiments comprise a polarizer up beam of the wave frontcorrector Examples of polarizers include: wire-grid polarizers andabsorptive polarizers.

Aberration sources may be characterized by Zernike Mode and theircontribution may be quantified by Zernike Mode strength. FIG. 12 shows agraph of Zernike Mode strength versus field angle for the referenceembodiment described in FIG. 11 and Table 3. It can be appreciated fromFIG. 12 that the primary contributor for wave front error for thefoveated optical system reference embodiment is Zernike Mode Noll indexNo. 6, or astigmatism, 1202. Comparatively, Zernike Modes Noll index No.1, 2, 3, 4, 7 and 8, 1204, produce very little aberrations. It should benoted that for other foveated optical system designs, other aberrationsources may be significant contributors to wave front error. Otheraberrations sources that may contribute to wave front error include:piston, tip, tilt, astigmatism, defocus, trefoil, coma, secondaryastigmatism, spherical, and other higher order Zernike Modes.

FIGS. 13A-13B depict astigmatism, or Noll-ordered Zernike modes 5 and 6.Referring back to FIG. 12, astigmatism, 1202, is an azimuthallyasymmetric aberration that in some optical systems gets worse withincreased field angle. In some embodiments, the wave-front correctordepicted in FIG. 11, may be configured to specifically correct forastigmatism. In this case, the astigmatic corrector consists of a liquidcrystal cell with two or more etched electrode layers in specificpatterns. An exemplary wave front corrector liquid crystal (LC) cell isdepicted in FIG. 14.

The LC cell in FIG. 14 consists of three transparent electrodes: apatterned electrode 1402, a floating electrode 1404, and a groundelectrode, 1406, with a liquid crystal solution, 1408, sandwichedbetween the floating and ground electrodes. Examples of different liquidcrystal materials include: Nematic, Smectic, Cholesteric, and Discotictypes. A transparent electrode may be constructed by depositing IndiumTin Oxide (ITO) onto a glass or plastic substrate, 1410. In someembodiments, one or more electrode layers are etched in specificpatterns in order to be able to produce a spatially varying potentialwithin the LC cell. A layer of a transparent insulator, such as SiO2,1412, is deposited on top of the patterned electrode, 1402. The SiO2 isetched to expose a small patch of each electrode, 1402. Electricalconductors, such as Nickel bus lines, 1414, are then connected to theelectrodes, 1402, 1404, and 1406, at these exposed patches.Preferentially, another layer of SiO2, 1412, is deposited on top of theNickel bus lines, 1414, to protect them from damage. In someembodiments, the electronically floating layer of transparentelectrodes, 1404, is deposited on top of the SiO2 layer, 1412. Thislayer contains an etched pattern of floating electrodes, 1404. Thefloating electrodes, 1404, are etched into a spatially varying pattern,like the patterned electrodes, 1402. Gaps between electrodes in afloating electrode layer can cause narrow regions within the LC materialto be less affected by electrical potential and thus to be not properlyoriented. The narrow regions of improperly oriented LC material causephase variations and introduce haze, or reduced contrast. In someembodiments, the etched pattern of the floating electrodes, 1404,coincide with the gaps in the patterned electrode, 1402. In someembodiments, the floating electrodes, 1404, are patterned such that eachfloating electrode is roughly half the width of the correspondingpatterned electrode, 1402, such that each floating electrode is wideenough to cover a gap between patterned electrodes, and at least ¼ ofeach of the patterned electrodes adjacent the gap. The floatingelectrode through capacitive coupling, will be at an average potentialbetween the two adjacent patterned electrodes it is overlaid upon. Thiseffectively multiplies the number of electrode regions (or steps)increasing the wavefront correctors resolution without increasing therequired number of electrical connections. The floating electrode layertherefore, acts to smooth the wave front correction, which decreases thescattered light from gaps between patterned electrodes. The Strehl ratiomay be used to indicate the effects of the floating electrode on theperformance of a wave front corrector. A typical wave front correctorhaving only patterned electrodes, with gaps between electrodes willyield a Strehl ratio of greater than 0.6, and typically greater than0.8. A wave front corrector additionally employing a floating electrodepattern will typically have a Strehl ratio greater than 0.7, andtypically greater than 0.9. In some embodiments, more than one floatingelectrode layer is used per patterned electrode layer to furtherincrease the resolution of the wave front corrector. For example, if onefloating electrode layer is used to bridge the gaps between a patternedelectrode layer, the number of electrodes, and resolution of the wavefront corrector doubles. To complete the LC cell, each electrode iscoated with a standard alignment layer, 1416, deposition and rubbingprocedure. Polyimide may be used for the alignment layer, 1416. In orderto efficiently produce wave front corrections that are specific tocertain aberration sources, transparent electrode layers are patternedaccording to the Zernike mode they are designed to excite.

In some embodiments, a wave front corrector is configured with one ormore layers, comprising transparent electrodes, 1502, that are patternedto excite Zernike mode Noll index Nos. 5 and 6. Transparent electrodepatterns that correct Noll-ordered Zernike modes 5 and 6 are depicted inFIGS. 15A and 15B. In some embodiments, transparent electrodes, 1502,are arranged in a pattern similar to that illustrated in FIG. 15A, inorder for the wave front corrector to selectively produce an obliqueastigmatism (Zernike mode Noll index No. 5). Likewise, in someembodiments, transparent electrodes, 1502, are arranged in a patternsimilar to that illustrated in FIG. 15B, in order for the wave frontcorrector to selectively produce a vertical astigmatism (Zernike modeNoll index No. 6). Electrical bus lines, 1504, are connected to roughlyevery third electrode.

In some embodiments, a wave front corrector is configured with one ormore layers, comprising transparent electrodes, 1602, which arepatterned to excite Zernike mode Noll index Nos. 2 and 3. A wave frontcorrector acting on Zernike mode Noll index Nos. 2 and 3, is similar toa liquid crystal wedge, and introduces a tip/tilt. Transparent electrodepatterns that correct Noll-ordered Zernike modes 2 and 3 are depicted inFIGS. 16A and 16B. In some embodiments, transparent electrodes, 1602,are arranged in a pattern similar to that illustrated in FIG. 16A, inorder for the wave front corrector to selectively produce a tip (Zernikemode Noll index No. 2). Likewise, in some embodiments, transparentelectrodes, 1602, are arranged in a pattern similar to that illustratedin FIG. 16B, in order for the wave front corrector to selectivelyproduce a tilt (Zernike mode Noll index No. 3). Electrical bus lines,1604, are connected to roughly every third electrode.

In some embodiments, a wave front corrector is configured with one ormore layers, comprising transparent electrodes, 1702, which arepatterned to excite Zernike mode Noll index No. 4. A transparentelectrode pattern that corrects Noll-ordered Zernike mode 4 is depictedin FIG. 17. In some embodiments, transparent electrodes, 1702, arearranged in a pattern similar to that illustrated in FIG. 17, in orderfor the wave front corrector to selectively produce a focus (Zernikemode Noll index No. 4). Electrical bus lines, 1704, are connected toroughly every third electrode.

In some embodiments, a wave front corrector is configured with one ormore layers, comprising transparent electrodes, 1802, which arepatterned to excite Zernike mode Noll index Nos. 7 and 8. Transparentelectrode patterns that correct Noll-ordered Zernike modes 7 and 8 aredepicted in FIGS. 18A and 18B. In some embodiments, transparentelectrodes, 1802, are arranged in a pattern similar to that illustratedin FIG. 18A, in order for the wave front corrector to selectivelyproduce a vertical coma (Zernike mode Noll index No. 7). Likewise, insome embodiments, transparent electrodes, 1802, are arranged in apattern similar to that illustrated in FIG. 18B, in order for the wavefront corrector to selectively produce a horizontal coma (Zernike modeNoll index No. 8). Electrical bus lines, 1804, are connected to roughlyevery third electrode.

In some embodiments, a wave front corrector is configured with one ormore layers, comprising transparent electrodes, 1902, which arepatterned to excite Zernike mode Noll index Nos. 9 and 10. Transparentelectrode patterns that correct Noll-ordered Zernike modes 9 and 10 aredepicted in FIGS. 19A and 19B. In some embodiments, transparentelectrodes, 1902, are arranged in a pattern similar to that illustratedin FIG. 19A, in order for the wave front corrector to selectivelyproduce a vertical trefoil (Zernike mode Noll index No. 9). Likewise, insome embodiments, transparent electrodes, 1902, are arranged in apattern similar to that illustrated in FIG. 19B, in order for the wavefront corrector to selectively produce an oblique trefoil (Zernike modeNoll index No. 10). Electrical bus lines, 1904, are connected to roughlyevery third electrode.

In some embodiments, a wave front corrector is configured with one ormore layers, comprising transparent electrodes, 2002, which arepatterned to excite Zernike mode Noll index No. 11. A transparentelectrode pattern that corrects Noll-ordered Zernike mode 11 is depictedin FIG. 20. In some embodiments, transparent electrodes, 2002, arearranged in a pattern similar to that illustrated in FIG. 20, in orderfor the wave front corrector to selectively produce a primary sphericalmode (Zernike mode Noll index No. 11). Electrical bus lines, 2004, areconnected to roughly every third electrode.

In some embodiments, a wave front corrector is configured with one ormore layers, comprising transparent electrodes, which are patterned toexcite Zernike mode Noll index Nos. greater than 11. These patterns arenot shown.

Wave Front Corrector Parameters Chart

Parameters related to wave front correctors are outlined in Table 4:

TABLE 4 Minimum Reference Maximum No. Patterned Electrode Layers (—) 110 25 (Not including floating electrode layers) No. Floating ElectrodeLayers (—) 0 10 50 Clear Aperture Diameter (mm) 0.1 mm 6 mm 50 mm No.Regions in Patterned Electrode (—) 2 33 1000 Note: Minimum width of atransparent electrode is typically about 200 nm Width of gap betweenelectrodes (um) 1  3 500 Note: Gap width is ideally as small as possibleNo. Zernike Modes Produced (—) 1 10 25 Thickness (mm) 0.1 mm ~3 mm 50 mmSwitching Frequency (Hz) <1 Hz 240 Hz 5 KHz Electrode PolarityAlternating Rate (KHz) <60 Hz 2 KHz 100 KHz Operable Wavelengths (nm)Achromatic within 400-700 nm range, and Monochromatic, and IR 700-1500nm. Location in Optical System At pupil plane, and outside of pupilplane Application Display Systems: Augmented Reality HMD, VirtualReality HMD, etc. Imaging Systems: Surveillance, Autonomous vehiclecamera systems. etc.

Specification for the wave front corrector, 1106, (depicted in FIG. 11)of the reference embodiment described above are included in Table 5,below:

TABLE 5 Aperture 6 mm Transparent Electrode Thickness 20 nm Etch WidthDefining Electrode Regions 3 μm SiO2 Thickness 130 nm LC Cell Thickness10 μm LC Material Merck 18349 Design Wavelength 550 nm (Center ofVisible Band) Max. Optical Path Difference (OPD) −3.3λ Electrode regionsper mode 33 OPD per region ~0.1λ

In some embodiments, electrode layers of the wave front corrector, 1106,(depicted in FIG. 11) are additionally configured with transparentresistors, 2102, as depicted in FIG. 21. In some embodiments, thetransparent resistors, 2102, are comprised of the same material as thetransparent electrodes, 2104, preferably indium-tin-oxide (ITO). Thetransparent resistors, 2102, bridge etch lines, 2106, which separateelectrodes, thereby electrically connecting adjacent electrodes, 2104,with a specified resistance. The reference embodiment comprisestransparent resistors, 2102, having a width of 4.5 μm and a length of 9μm. The electrical resistance provided by the transparent resistors is afunction of the size of the transparent resistor, 2102. The thickerwider or larger in cross-sectional area the transparent electrode is,the lower the electrical resistance provided. Likewise, the longer thetransparent electrode is the greater its electrical resistance. In someembodiments, the transparent resistors allow for multiple electroderegions, 2104, to be driven with a single conductor (depicted in FIGS.15-20).

In some embodiments, wave front correctors are provided that havesufficient etched electrode patterns to correct Noll Order Zernike Modes2-8. Other modes may also be corrected. For example, the referenceembodiment described above is configured with a wave front correctorhaving sufficient etched electrode layers to correct Noll Order ZernikeModes 2-11. Optical performance for the reference embodiment isdescribed below. The 15 Zernike Modes in the pupil plane are shown inFIG. 22A. The 15 Zernike Modes in the retinal plane are shown in FIG.22B. The 15 Zernike Modes shown in FIGS. 22A and 22B comprise: Piston,2201; tip, 2202; tilt, 2203; defocus, 2204; oblique astigmatism, 2205;vertical astigmatism, 2206; vertical coma, 2207; horizontal coma, 2208;vertical trefoil, 2209; oblique trefoil, 2210; primary spherical, 2211;vertical secondary astigmatism, 2212; oblique secondary astigmatism,2213; oblique quadrafoil, 2214; and vertical quadrafoil, 2215.

FIGS. 23A-23C contain corrected and uncorrected point source functionimages for the reference embodiment described above. Full sizedcorrected and uncorrected point source function images corresponding tothose shown in FIGS. 23A-23C are shown in FIGS. 60-93. FIGS. 24A-24Ccontain corrected and uncorrected wave front images for the referenceembodiment described above. Full sized corrected and uncorrected wavefront images corresponding to those shown in FIGS. 24A-24C are shown inFIGS. 94-127. It can be seen that uncorrected image quality decreaseswith increased field angle. Image quality at a given field angleincreases as the wave front corrector is controlled to correct aberrantmodes at a given field angle. A presently corrected area within thefield of view is defined as a region of interest. The region of interesthas a size, determined by the foveated optical system design.

FIGS. 25A-25B are plots that show resolution limit as a function offield angle for a region of interest using the reference embodiment of afoveated optical system. In reference to FIGS. 25A-25B, resolution limitis defined as a spatial frequency where modulation transfer function(MTF) drops to 10%. The resolution limit in FIG. 25A is shownproportional to Nyquist sampling, where 2*Nyquist is defined as theideal spatial frequency. The resolution limit in FIG. 25B is the samegraph as FIG. 25A except that the resolution limit has been normalized,so that the highest resolution limit is equal to one (1.0). It can beappreciated from the graphs that there is no significant change to theresolution limit under 4° field angle, at which point the resolutionlimit decreases. It can be seen from FIG. 25B that at a field angle ofapproximately 4.5° the resolution has dropped 10%. Therefore, the regionof interest can be said to have a size of approximately 9° total, or±4.5°. Performance outlined in FIGS. 25A-25B was achieved withoutadditional symmetric aberration correction by the wave front corrector.That is to say, focus and spherical aberrations were left uncorrected bythe wave front corrector in the determination of these plots. Withadditional symmetric aberration correction provided by the wave frontcorrector, the region of interest will only increase in size. In someembodiments, the region of interest is as small as 1°, although a regionof interest of at least 2° is preferred, and regions of interestsgreater than 15°, 30°, 45°, 60°, and 90° are possible depending onresolution limits. FIG. 25A shows a maximum resolution limit of justunder 1.2*Nyquist. In other embodiments the foveated optical systempreforms with a resolution limit greater than 1.2, 1.3, and 1.4*Nyquist.In still other embodiments, the foveated optical system has beenconfigured such that the resolution limit of the system is reduced tobelow 1.0, 0.9, and 0.8*Nyquist in order for the region of interest sizeto be greater.

Referring to FIG. 26, in some embodiments, the region of interest may beselectably moved to any location within the field of view, 2604, byintroducing wave front changes with the wave front corrector. Asimplified example is provided by the reference embodiment, where thefoveated optical system error is largely from a single mode at a certainfield angle. Referring back to FIG. 12, it can be seen that at largefield angles astigmatism, 1202, specifically Zernike Mode Noll index No.6, or vertical astigmatism, provide the greatest wave front error. Inorder to effectively move the region of interest to a position, 2606,having a large field angle, such as (Y₁, 0), a wave front change isintroduced to correct the astigmatism at that location. Referring toFIG. 26, a second region of interest, 2608, is shown located in bothCartesian and Polar Coordinates. And, a third region of interest, 2610,is shown at an origin, or the center of the field of view, 2604. Asdescribed above, a wave front change is introduced by altering the indexof refraction within a liquid crystal layer in a specific pattern. Thisis achieved by providing patterned transparent electrodes adjacent aliquid crystal layer with appropriate electrical potentials. In order tocorrect large vertical astigmatic modes, electrical potential must beprovided to patterned electrodes, such that a wave front change isintroduced. The wave front change acts on the light similarly to anappropriately powered cylindrical lens to correct the astigmatism.

Referring to FIG. 27, an electrode pattern for correcting verticalastigmatism is shown above, and an optical path difference plot (OPD) isshown below. The OPD plot shows the astigmatic error, 2702, along thevertical axis of the wave front corrector. The electrode pattern isarranged according to correct vertical astigmatism. Boundaries of theelectrodes are contour lines of a vertical astigmatism wave front.Vertical astigmatism is a part of a Noll indexed Zernike basis set, andtherefore a part of an orthonormal basis set. Because the electrode ispatterned according to the mode of an orthonormal basis set, the wavefront corrector may correct a single mode (astigmatism), withoutsignificantly impacting other modes. The OPD plot, 2704, additionallyshows the wave front correction, 2706, introduced by the wave frontcorrector to correct the vertical astigmatic aberration. The electrodepattern is such that the highest frequency of electrodes are locatedwhere vertical astigmatism wave front changes are steep. Likewise, inthe center of the wave front, where vertical astigmatic errors changemore slowly, fewer electrodes are present. Thus, the residual wave fronterror is minimized, as the number of electrodes are conserved. An idealwave front change to correct the vertical astigmatism is an equal andopposite wave front to the vertical astigmatism. FIG. 27 shows that thewave front change produced by the wave front corrector is not a smoothline, like the vertical astigmatism, but is comprised of a number ofsteps. Each step in a wave front change corresponds to an electrode. Thedifference between a wave front correction produced by the wave frontcorrector and an ideal wave front correction required to undo the wavefront error is the residual wave front error. The residual wave fronterror, is less than 40%, and preferably less 20%, 10%, and 5%.

FIG. 28 illustrates an ideal or desired wave front change, 2802, and astepped or actual wave front change, 2804. The difference between thestepped wave front change, 2804, and the ideal wave front change, 2802,is depicted as residual wave front error, 2806. More generally,deviations between a wave front change produced by a wave frontcorrector, 2804, and a corresponding ideal wave front change, 2802, canbe defined proportionally. A wave front change, 2804, is considered 90%of an ideal wave front change, 2802, when the two changes differ by 10%.Or said another way, a wave front change, 2804, that corrects wave fronterror yielding a 10% residual wave front error, 2806, is considered 90%of an ideal wave front change, 2802, that corrects the entire wave fronterror.

Residual wave front error from the reference foveated optical systemembodiment is qualitatively shown in corrected PSF and wave front imagesabove. For the reference embodiment the measured Strehl ratio wasgreater than 96%.

FIG. 29 shows a wave front corrector electrode pattern arranged toproduce spherical wave front changes. Divisions between electrodes areetched along contour lines of a spherical wave front. Contour linestypically are traced where the wave front is a constant value. Forexample, equi-phase contour lines trace over points of constant phase.In some embodiments, it is possible, but likely disadvantageous, to havethe divisions of the electrodes located on lines that are traced wherethe wave front value is within a certain range, but not exactlyconstant. For example, in some embodiments electrode divisions aretraced where wave front values are within 50%, 30%, 20% or 10% of aconstant value. An exemplary spherical wave front electrode pattern maybe created having electrode divisions, which for manufacturing reasons,are rectilinear, so long as the divisions are traced over areas where aspherical wave front having a near constant value. In such non-idealembodiments, wave front changes produced by the wave front correctorwith the rectilinear spherical wave front pattern, will have greaterresidual wave front error.

An optical path difference (OPD) plot, 2902, of a spherical aberration,2904, along the horizontal axis of the wave front corrector is shownbelow in FIG. 29. A wave front correction, 2906, opposite the sphericalaberration is also shown. The correction, 2906, on the OPD plot, 2902,shows that narrow electrodes are densely packed where spherical wavefronts are steepest, and less dense wider electrodes are located wherespherical wave front changes are slow. Because a spherical wave frontmay be defined as a part of an orthonormal basis set, it is possible tomake spherical wave front corrections without significantly changingother wave front modes. It can be generally appreciated from the OPDplot, 2902, the wave front corrections, 2906, of the sphericalaberrations, 2904, result in a residual wave front error less than 40%,and typically less than 10%.

Wave front changes required to correct aberrations at certain locationsmay be determined from optical modeling, as described above and inreference to FIG. 12. Similarly, the electrode potential values requiredto produce the required wave front changes may be determined from theoptical and electrical properties of the liquid crystal material used,as well as the known dimensions of the pattern of the transparentelectrodes. In some embodiments, a look-up table is used to store theelectrode potential values required to correct wave front errors at anumber of different locations within the field of view, and effectivelymove the region of interest to those locations. In some embodiments,electrode potential values are interpolated from look-up table values tomove a region of interest to a desired location, which is not collocatedwith a stored location within the lookup table. Alternatively, thelook-up table includes a sufficient number of location entries, suchthat the desired location is always within a region of interest locatedat a nearest location entry within the lookup table. In anotherembodiment, electrode potential values are determined from functions ofthe position of the region of interest, in either Cartesian or polarcoordinates.

The electrode potentials are supplied by a liquid crystal controller. Anexample of an off the shelf, single-channel liquid crystal controller,which is suitable to provide an electrical potential to one or moreelectrodes is: Thorlabs Part No. LCC25. Typically, a liquid crystalcontroller will provide electronic signals, which alternate in polarity.For example, Thorlabs Part No. LCC25 alternates signal polarity at 2Khz. Ideally, the liquid crystal material will respond linearly tochanges in electrode potential. The reference embodiment comprises aliquid crystal material that responds linearly to potential in the rangeof 1-2.5V.

In some embodiments, a liquid crystal controller comprises: amicroprocessor, and a multi-channel digital-to-analog converter (DAC). Amulti-channel DAC chip provides an analog signal to multiple electrodes,in response to a digital input value provided by the microprocessor. Anexemplary multi-channel DAC is National Instruments Part No. NI 9264,which has 16 channels that can be simultaneously updated at a rate asfast as 25 KHz. Each of the exemplary electrode patterns describedabove, contain 33 electrodes and therefore require 33 potential levels.However, because the exemplary electrode patterns comprise transparentresistors, fewer than 33 potential levels may be provided, per electrodepattern. The above described electrode patterns typically have three (3)or more electrodes being supplied by one (1) conductor that is connectedto the liquid crystal controller. The microprocessor is configured tosend digital input signals to the DAC, such that the polarity of theelectrode signals is alternated at a rate of about 2 Khz. As mentionedabove, in some embodiments the liquid crystal controller operates inresponse to other systems, such as: an eye tracking system, or otherinterfaces, such as keyboard, mouse, or joystick. In such embodiments,additional software is configured to modify the digital signals sent tothe DAC in order to provide the required electrode potential values toachieve the desired wave front changes.

In many cases, a foveated optical system which is incorporated into ahead mounted display, moves the region of interest in response to thetracked orientation of a user's eyes, such that the user's gazecoincides with the region of interest. As mentioned above a human eye,because of its structure has a limited central field of view, over whichit senses light at a high resolution. Outside this central field ofview, human vision is poor. Embodiments of foveated optical systems maytherefore correct wave front error, and provide a high resolution imageto the eye within its central field of view. In some embodiments, aneye-tracking system is incorporated in order to provide feedbackregarding the position of a user's eye.

Accommodating Projector System

In order to generating stereoscopic accommodated, augmentation imageryand project it onto retina's of a user's eyes, a HMD may incorporate anAccommodating Projection System. An Accommodating projector is aprojection system that's purpose is to render an augmentation imagewhere each pixel in a scene is assigned a unique RGB color value and afocal depth associated with its distance along a line-of-sight (LOS) ofa user's eye. An embodiment of an accommodating projection system for auser's left eye is diagrammed in FIG. 30A (right eye is just a mirroredversion of the same design).

Referring to an embodiment of an accommodating projector systemillustrated in FIG. 30A, color encoded light from a projector source,3002, is reflected off the surface of a MEMs mirror, 3004. The MEMsmirror raster scans an image of the augmentation scene onto an imageplane aperture stop of the system, 3006. The light is further imaged bya lens, 3008, and then a variable lens, 3010. Referring to FIG. 30B, adisplayed image, 3012, results that comprises a multitude of resolutionelements or pixels, 3014. Raster scanning, 3016, of a displayed image,3012, may be performed in such a manner that the image is displayed bymoving a projected beam across the aperture stop,3006, at a fasthorizontal rate and a slower vertical rate as depicted in FIG. 30B.According to some embodiments, a light generator that comprises any of afamily of 2D pixilated display technologies is used to project theaugmentation imagery. Examples of 2D pixilated display technologiesinclude: liquid crystal on silicon (LCOS) 2D microdisplay, and OLEDmicrodisplay. According to some embodiments, light passing through theimage plane aperture stop, 3006, is collimated by a lens, 3008, and avariable lens, 3010. The variable lens may be a liquid crystal lens,3008. The liquid crystal lens (LCL) shapes the wave front of a beam asit passes through the liquid crystal lens. An effective focal length ofthe LCL is selectively controlled in response to an applied current. Aschematic of an LC lens is drawn in FIG. 31.

Referring to an embodiment of a liquid crystal lens, 3100, shown in FIG.31, liquid crystals, 3102, are sandwiched between two plates of glass,3104A and 3104B, and are connected with electrical conductors, 3106,allowing a potential to be applied across them. The liquid crystals inan LC lens have the property that they tend to align with an appliedpotential. When a light is passed through the liquid crystals, the lightexperiences an index of refraction that varies with a potential appliedto the liquid crystals. With multiple concentric transparent electroderings, 3108, are used to apply an azimuthally symmetric potential acrossthe device. By applying different potentials to each ring the index ofrefraction changes radially, shaping the wave front into a lens. In someembodiments the electrodes of an annular spatial light modulator arecreated by depositing Indium Tin Oxide (ITO) onto a glass substrate. TheITO glass is transparent to visible wavelengths and is therefore ideallysuited for this application.

An embodiment of an accommodating projector that projects augmentationimagery having an added focal depth is illustrated in FIGS. 30A-30B. Theadded focal depth shapes a wave front of light, which comprises thedisplayed augmentation imagery. Shaping the wave front of the projectedlight as if it were emanating from the real world allows a user's eye toaccommodate the stereoscopic imagery. A human eye can only focus at asingle focal depth at a time. But, the human eye can discernapproximately a dozen focal depths in its field of vision. Therefore, insome embodiments the variable lens, 3008, has a focal depth range thatcomprises about 12 focal depths. A process of accommodating projectionmay be accomplished by at least two methods involving projection andwave front shaping:

A first method for accommodating projection is a fast-rate multiplexingmethod. This method comprises refreshing each pixel at a rate that is aninteger multiple (N_(z)) of your desired refresh rate (fast-rate). Theinteger multiple, N_(z), is a number of focal depths to be displayed.The number depends on several design factors, but in at least someembodiments, a full depth range of an eye is not required. Instead onlya single focal depth, which the user is currently focused upon, and anumber of nearest focal depths within the central FOV are projected. Forexample, a total number of projected focal depths may be equal to 1, 2,3, or 4 (N_(z)=1-4 depth levels). After each refresh of the image, thefocal depth is changed and the image is refreshed with another focaldepth. The user's eyes now integrate over these several realizations ofthe image. As an example, a fast-rate multiplexing accommodatingprojector system may project four focal depths simultaneously at adesired refresh rate of 120 Hz. In this example, N_(z)=4 and theprojector system refreshes each pixel at a rate of 480 Hz.

A second method for accommodating projection is a slow-rate multiplexingmethod. It can be seen from the above example that the fast-ratemultiplexing method can include a high projector refresh rate toaccomplish accommodating projection. The slow-rate multiplexing methodcomprises a lower required pixel refresh rate. The slow-ratemultiplexing method maintains the refresh rate, and updates the focaldepths at an accommodation update rate that allows each multiple focaldepth to be projected within an integration time of a human eye. Theprojector update rate is selected to ensure the pixels are updatedwithin the integration time of the eye. Therefore, if the accommodationupdate rate of the accommodating lens is faster than the projectorupdate rate, then the desired human eye integration time will be met.Referring back to our previous example, the slow-rate multiplexingmethod comprises a 120 Hz pixel refresh rate with a 120 Hz accommodationupdate rate. In reference to the slow-rate multiplexing method, theprojector frame may be updated at projector update rate of 30 Hz(slow-rate). Each pixel is given 4 focal depth values to produce thecorrect weighted focal depth. A refresh rate, such as 120 Hz, maintainslow flicker. A 120 Hz refresh and accommodation update rate, coupledwith a projector update rate of 30 Hz allows 4 focal depths per frame ofaugmentation imagery projected.

In some embodiments, there is no predefined level of image quality foran athletic HMD to achieve athletic performance. However, HMD devicesgenerally are expected to deliver imagery of a certain specification.Ideal HMD display specifications, which are not necessarily achieved inall embodiments, are: Provide a display resolution that displays asmallest resolvable resolution element that is no larger than aresolution element of an average human eye. Provide a display frame ratethat renders the update image frames at a rate that is imperceptible tothe human eye, for example greater than 10 Hz. And, provide a displayrefresh rate that refreshes pixels at a refresh rate that isimperceptible to the human eye.

Display Resolution

A display resolution is the smallest applicable resolution element. Inorder to achieve ideal imaging the resolution element would beapproximately the same size as the rods/cones detectors in the eye. Thisis known as retina-limited imaging. In some embodiments of an A-HMDsystem, a resolution provided to a user varies within a range ofpossible values that include retina-limited imaging.

Display Refresh Rate

A display refresh rate is a refresh rate at which a pixel is updatedwith a new value. The eye's natural integration time varies depending onthe type of cell. Rods have an integration time of approximately 100milliseconds and cones have an integration time of 10-15 milliseconds.If a pixel is not refreshed before the next integration begins, a viewermay see an unintended amplitude flicker. For gaming applications,refresh rates are typically in a range of about 60-120 Hz, or greater.

Display Frame Rate

A display frame rate is a rate at which frames are updated for display.Frame rate and refresh rate may differ, because pixels may be refreshedmore often than a frame is updated. In some embodiments a refresh rate,which is greater than a frame rate, is employed, causing individualpixels to be refreshed multiple times with a single value. A human eyehas an effective frame rate of approximately 11 Hz. Frame rates aretypically in a range of about 24-30 Hz, or greater.

In some embodiments an A-HMD will display augmentation imagery withdepth accommodation. Ideal focal depth control specifications, which arenot necessarily achieved in all embodiments, are: Provide a focal depthrange that shapes a wave front of projected imagery over a focal depthrange that is consistent with the accommodation range of an averagehuman eye. Provide a sharp field of view (FOV) that renders a sharp FOVover the user's fovea, enabling human foveal vision of augmentationimagery. Provide a focal shift rate that shifts focal depths of theprojected imagery at a frequency imperceptible by the human eye. And,provide a focal correction range that provides focal correction, whichspans the total range (2σ or 96%) of human inset depths.

Focal Depth Range

Each resolution element of a virtual scene may be rendered at aparticular focus. A focal depth range is a range of possible focaldepths that may be applied to a ray. In some embodiments, the focaldepth is applied by shaping the wave front of the ray, with anadjustable focus optic. The focal depth range determines the depth ofthe virtual scene. Rays that converge on the outer edge of the retinawill experience significant aberrations due to the imperfections in thehuman visual system. This will limit the effective area of a scene thatmay be brought to sharp focus. FIG. 32A shows a scene, 3202, imaged at anumber of focal depths, 3204.

Sharp FOV

Sharp Field of View (FOV_(S)), 3206, is an angular area over which raysmay be brought into sharp focus by an eye, as shown in FIG. 32B. Inactuality, the sharpness of the image, as measured by a system's opticaltransfer function (OTF), degrades with radius from a center of the fieldof view. The Sharp FOV may technically be defined as a solid angle overwhich rays may be brought into sharp focus; where sharp focus maytechnically be defined as at least 96% an ideal system modulationtransfer function (MTF). If the system's image quality degradesaccording to a Gaussian function of the radial angular distance, theSharp FOV may be defined as the 2□ point of the field degradationGaussian profile. Non-sharp FOV, 3208, is the FOV outside the sharp FOV,3206.

Focal Shift Rate

A focal shift rate is a rate at which focus of a ray may be changed.Focus shifting devices tend to have a linear and nonlinear operationalregime. In the linear regime, the focus shift rate is linearly relatedto the size of the focal shift, implying that in the linear regime thefocal shift rate is constant. However, in the non-linear regime, thefocal shift rate is not linearly related to the focal shift size and thefocal shift rate may grow increasingly fast as the size of the focalshift is increased.

Focus Correction Range

Some embodiments of a HMD are capable of performing an auto-focusoperation to determine an eye's focusing characteristics. A focal rangemay be applied during the auto-focus process, and is known as a focuscorrection range. It is desirable to create a system with a focalcorrection range large enough to encompass the majority of users. Anexample of an HMD comprising an autofocus system is explained below ingreater detail.

A controller determines one or more focal depths, based upon a distancealong a user's line of sight that displayed imagery should be perceivedat. In some embodiments, a focal depth is determined based uponeye-tracking data from one or more eyes. In one embodiment, eye-trackingdata: comprising a gaze vector; as well as, head-tracking data, andrender data are used by the controller to determine a focal depth. Therender data comprises information that defines: content and location ofan augmentation scene. The defined location of the augmentation scene isin global (real world) coordinates. The head-tracking data comprisesinformation defining a position and a pose of the user's head in globalcoordinates. The controller determines, based upon the head-trackingdata and render data, a distance along the gaze vector at which theaugmentation imagery should be perceived. A focal depth is derived fromthe determined distance along the gaze vector.

According to some embodiments, eye-tracking data, comprising a gazevector, from both eyes may be used to determine a focal depth. Forexample, a controller determines a point of intersection between gazevectors from a right and a left eye of a user. A focal depth is derivedfrom the point of intersection of the two gaze vectors. An exemplaryeye-tracking system is explained below, although other versions ofeye-tracking systems may be incorporated interchangeably.

Eye-Tracking System

Some embodiments of an HMD comprise an eye-tracking system, in order toprovide eye-tracking data in response to a position of a user's pupil.Any method of eye tracking that provides feedback in response to theposition of a user's pupil may be implemented in a HMD. For example, insome embodiments a video-based eye tracker is implemented. An embodimentcomprising a video-based eye-tracker is illustrated in FIG. 33.Referring to FIG. 33, a diffuse light source, 3302, illuminates an eye,3304, with a diffuse light, 3306. A camera, 3308, measures lightreflected from the eye's surface. In some embodiments, an infrared (IR)diffuse light source and an IR camera are implemented in conjunction.Some embodiments further comprise a beam combiner, 3310, and a beamlauncher array, 3312, as described above.

FIGS. 34A-B illustrates reflection of the diffuse light as measured by avideo-based eye-tracking system like that in FIG. 33. A bright pupil,3402, indicates that the diffuse light source is generally coaxial withthe optical path of the eye (not shown). A bright pupil measurement isillustrated in FIG. 34A. Where the diffuse light source is generallynon-coaxial with the optical axis of the eye, a dark pupil, 3406, ismeasured having a bright corneal reflection, 3408, as shown in FIG. 34B.A threshold may be applied to the measurement in order to determine onlya portion of the measurement that corresponds to the pupil.Determination of the portion of the measurement that corresponds to adark pupil is an inverse calculation of that for a bright pupil. Acentroiding system employing a centroiding algorithm, such as that inEquation 1, determines a center of the pupil.

$\begin{matrix}{\overset{harpoonup}{R} = {\frac{1}{I_{tot}}{\int_{A}{( {{I( \overset{harpoonup}{r} )} \cdot \overset{harpoonup}{r}} ){dA}}}}} & (1)\end{matrix}$

The position of the pupil is then determined, based upon the center ofthe pupil as measured, and a known orientation of the camera withrespect to the user's eye. In embodiments employing a dark pupilmeasurement as illustrated in FIG. 34B, a vector between the center ofthe pupil and the corneal reflection, 3408, may also be used to deducethe gaze direction. In some embodiments, eye-tracking data comprises theposition of a user's eye. In other embodiments, eye-tracking datacomprises a gaze vector. The gaze vector is determined based upon theposition of the user's pupil and represents a line of sight (LOS) of theuser.

Error in the estimated pupil center position as compared to the actualpupil center position is referred to as the Pupil Position error, and isillustrated in FIGS. 35A-B. A pupil position error may be calculated asa positional error, having units such as millimeters. The pupil positionerror may, also be calculated as an angular error, having units such asmilliradians. FIG. 35A illustrates an orientation of an eye, 3502, interms of angles of rotation about coordinate axes. The pupil positionerror is depicted in FIG. 35B by showing an estimated orientation of aneye, 3504, as well as a true orientation of an eye, 3506. Pupil positionerror has a potential to affect the accurate function of HMDs comprisingeye-tracking. The amount of pupil position error that is acceptable tosuch HMDs and HMD subsystems is dependent upon the perception of theuser. Ideal eye-tracking specifications, which are not necessarilyachieved in all embodiments, are: An eye-tracking rate that obtainseye-tracking updates at a rate that is imperceptible to the human eye.An eye-tracking accuracy that obtains eye-tracking updates that areaccurate to within a specified tolerance. And, an eye-tracking precisionthat obtains eye-tracking precision, which allows for corrections thatare unresolvable by an average human eye.

Movements of an eye that result from a change in viewing direction arelarge and are measurable by an eye-tracking systems, as described above.In some versions of an HMD, smaller eye movements caused by jitter, or aslight movement of an HMD relative a user's eyes are not measured by aneye-tracking system. Instead a jitter sensor takes measurements thatcorrelate with smaller eye movements. The smaller eye movements are thencorrected for by an augmentation stabilization system. In someembodiments, eye-tracking system specifications are loosened, as othersystems allow for an increase in pupil position error. For example, anaccommodation projector system that projects augmentation imagery atgreater than three focal depths allows the HMD to more generallyestimate the user's focal depth, and thus gaze vector. Likewise, asystem that provides high resolution imagery over a FOV greater than auser can perceive in high resolution, can include a lower-accuracyeye-tracking system.

Eye-Tracking Rate

Tracking the position of an eye's pupil is known as eye-tracking. A rateat which an eye-tracking system can track the motion of an eye's pupilis an eye-tracking rate. In some embodiments, an A-HMD obtainseye-tracking data at a frequency no less than a projector update rate,also referred to as a frame rate or a rate at which a projected image isupdated. Therefore, typical eye-tracking rates are greater than 30 Hz,and generally are about 60 Hz, 120 Hz, 240 Hz, 480 Hz or greater.

Eye-Tracking Accuracy

In some embodiments of an HMD, eye tracking accuracy limits how wellaugmentation imagery will be placed in a scene. That is to say howaccurate the location of the augmentation imagery as perceived by theuser is to the location targeted by the HMD. Eye-tracking accuracy has astronger effect on augmentation imagery that is to be perceived ascloser to a user than imagery that is to be perceived at a distance.This is because a positional error leads to an angular error given by:

$\begin{matrix}{{EyePose}_{error} = \frac{dx}{z}} & (2)\end{matrix}$

Eye-Tracking Precision

Eye Tracking Precision refers to, how consistently the position of thepupil is tracked. Even if there is a nominal offset error that misalignsthe actual and estimated pupil positions (inaccurate), it is possible tomeasure the same position repeatedly (precise). Eye tracking precisionmay be given by the rate of error change as a function of time:

$\begin{matrix}{{\Delta \; {EyePose}_{error}} = {\int_{0}^{\tau}{\frac{{dx}(t)}{z(t)}{dt}}}} & (3)\end{matrix}$

Where τ is an integration time dictated by the eye-tracking rate.

The Augmentation Stabilization Control System

For athletic applications, it is not enough to render the correctaugmentation imagery; it should be rendered in the right place. Thatmeans, for example, that it should be immune to the motion of a user toa degree greater than previous or currently available HMD devices. Ifthe update rate of the projector system, or the frame rate, is 30 Hz forinstance, then head motion faster than this rate will appear to move theaugmentation imagery from the desired location within the FOV of auser's eye. An augmentation stabilization system typically addresses twotypes of jitter, optical jitter, 3602, and augmentation jitter, 3608.Optical and augmentation jitter are shown in FIGS. 36A-B.

As illustrated in FIG. 36A, optical jitter, sometimes referred to asprojection jitter, 3602, is movement of an optical system of a headmounted display in relation to a user's eye, 3604. In some versions,optical jitter is controller with firm HMD mountings. Augmentationjitter, 3608, as illustrated in FIG. 36B is movement of the user and theoptical system of the HMD with respect to an image source, 3606. Inrespect to augmented reality display, the image source, 3606, may be thereal world. This occurs because the augmentation imagery is generated ina local eye coordinate system, but is made to appear as if it emanatedfrom the image source, 3606, that may be defined in a global coordinatesystem. The augmentation jitter, 3608, will appear to a user as a jitterthat reduces the resolution of the augmentation imagery or worse yet,ruins the immersive experience by highlighting to a user what is realand what is augmented. Therefore, it is desirable to ensure thataugmentation imagery appears stable and locked in place in the globalcoordinate system. This may be accomplished by adding an augmentationstabilization system to the optical chain. An embodiment of anaugmentation stabilization system is depicted in FIG. 37. A first lens,3702, collimates light, 3704, emitted from an augmentation image to forma pupil plane coincident with the vertex of a fast steering mirror,3706. A second lens, 3708, re-images the pupil plane coincident with thefast steering mirror onto a pupil alignment mirror, 3710. The pupilplane coincident with the pupil alignment mirror axis may be re-imagedonto the eye or another intermediate pupil plane by a third lens, 3712.Orientation changes to a user's head cause the user's eyes to changeorientation and shift position. The change in orientation and positionof a user's eyes is defined as eye jitter. If the orientation changesare small (dθ<<1), perspective changes to the augmentation imagery arenegligible, and only image translations need be corrected. Eye jitter isillustrated in one dimension in FIG. 38.

The image displacement, dx, 3800, is given by the equation:

dx=(Z _(p) +R _(eye))·tan dθ  (4)

Where Z_(P), 3802, is a distance to the augmentation image point P,3804; R_(eye), 3806, is a distance from the rotation axis of the head tothe rotation axis of the eye, and dΘ, 3808, is a rotation jitter.Equation (4) may be applied to both vertical and horizontal eye jitter,where dΘ is either the rotation about the horizontal or vertical axisrespectively.

Jitter data is collected from a jitter sensor, at a jitter sensing rate.The jitter is then corrected by the augmentation stabilization system atan augmentation stabilization rate. During jitter correction, theaugmentation stabilization rate can be high, typically greater than theframe rate of the projector, and often greater than the refresh rate ofthe projector. Therefore, the jitter sensing rate should be of asimilar, or faster, rate than the jitter correction rate. In someembodiments, the jitter sensor measures movements of a user's head, inorder to provide jitter data at high rates. A controller that isprovided with the jitter data estimates the position of a user's eyes,and adjusts optical components to correct the imagery accordingly.

In some embodiments, a fast steering mirror (FSM), 3706, is used toshift the location of the imagery in the image plane by adding tip andtilt phase gradients to the pupil plane (Fourier shift theorem). Inorder to shift the imagery correctly, the distance to the augmentationimagery must be known or approximated. In some embodiments, theaugmentation stabilization system and the FSM, 3706, are updated at arate that is at least as fast as the refresh rate of a projector, whichis generating the imagery. Typically, MEMs type FSMs are updated at kHzrates and the projector is refreshed at a rate greater than 30 HZ andmore typically between 120-480 Hz. Eye jitter at frequencies below theupdate rate of the projector system, which is generally greater than 11Hz and typically about 30 or 60 Hz, may be corrected by rendering theimagery shifted in location.

Eye Jitter Sensor

In some implementations, correcting augmentation jitter includes sensingand measuring augmentation jitter. As stated above, augmentation jitterresults from head motion that imparts a change to the pose of a user'seyes. Therefore, in order to measure augmentation jitter you need tomeasure the motion of a user's eye. An inertial measurement unit (IMU)is a device for measuring the pose of an object. An IMU works bydetecting changes in a rate of acceleration with one or more linearaccelerometers and by detecting changes to rotational attributes (e.g.roll, pitch, and yaw) by using one or more gyroscopes. IMUs may alsoincorporate magnetometers or angular accelerometers as well. IMUs areable to provide measurements at rates specified by an augmentationstabilization system. In one embodiment, a controller continuallycalculates the pose of an object being measured by an IMU by integratingchanges in acceleration to calculate the current velocity. Then, thecontroller integrates the velocity to obtain an estimate of the object'sposition and orientation. Inertial guidance systems that incorporate anIMU typically suffer from accumulated error. Some embodiments includejitter sensors that comprise accelerometers in perpendicularconfiguration (x-y). In some versions, absolute positional error isminimized with periodic updates from an absolute geo-positioning sensor,such as a GPS head tracking system. In a further example, movement ofthe eye can be determined by a snsor that measures forces exerted on abody (e.g., a known mass) that shares translational movements with theeye. For example, a sensor practicing this technique can be included inan HMD, and because the HMD moves with the wearer's head, the sensorshares such movements with the eye.

Stability Function

Stability can be divided into two types: optical stability andaugmentation stability. Optical jitter, 3602, as depicted in FIG. 36A,results from the apparent motion of an optical system between to auser's eye, 3604, and the environment, 3606. Augmentation jitter, 3608,as depicted in FIG. 36B, results from the apparent motion of the userand HMD with respect to the global coordinate system. Ideal HMD imagestability specifications, which are not necessarily achieved in allembodiments, are: Provide augmentation stability by placing augmentationimagery in a fixed global position with a precision that is smaller thanthe resolving power of a human eye. And, provide optical stability byattaching the HMD to the user, such that the variation in position doesnot exceed the resolving power of the human eye.

Augmentation Stability

The augmentation stabilization system senses augmentation jitter, andcorrects augmentation jitter faster than is perceptible to a human eye.Sensing of augmentation jitter is achieved by head-tracking systems, andin some embodiment's eye-tracking systems. Correcting of the position ofaugmentation imagery is achieved through an optical system thattranslates the augmentation imagery at the user's pupil. Augmentationstability is, thus governed by an augmentation stabilization rate. Anaugmentation stabilization rate is an update rate at which augmentationjitter is stabilized. This update rate is itself throttled by a jittersensing rate and a jitter correction rate. The jitter sensing ratecomprises a rate at which jitter is measured, such as: an eye-trackingrate, an IMU sensing rate, and/or a head-tracking rate. The jittercorrection rate comprises a rate at which the augmentation imagery istransformed, such as: a rate of adjustment of optical elements, likefast steering mirrors or adjustable gradient optics. Regardless, someresidual augmentation jitter, or post-stabilization augmentation jitter,remains after augmentation stabilization. The post-stabilizationaugmentation jitter can limit the performance of the system. The smallerthe residual jitter than the more “glued” to the real world the virtualcontent will seem. Therefore it is highly desirable to increase thestabilization system's rejection bandwidth (range of frequencies thatare rejected by the stabilization system) in order to minimize the poststabilization residual jitter. Residual jitter is decreased by increasedaugmentation stabilization rates.

Optical Stability

Optical stability is measured in a similar fashion to augmentationstability. However, they differ in one key way: augmentation jitter isunavoidable whereas optical jitter may be reduced by design. Therefore,in some embodiments, optical stability is achieved throughopto-mechanical considerations, such as: the use of a stiffer structure.Additionally, a wave front control system may be employed to reduceoptical jitter.

Head-Tracking System

In some embodiments, an HMD may comprise a head-tracking system thatprovides periodic updates, relating to the position of a user's head,keeping a position error low. Head tracking is a process of estimatingthe position and orientation (pose) of a user's head. This is depictedgraphically in FIG. 39A. FIG. 39A shows a global coordinate system(x,y,z), 3902, and a local coordinate system (x′, y′, z′), 3904.Technically, global positions of a user's eyes are most relevant to anHMD controller. The positions of the eyes are fixed with respect to thehead, and thus the center position of the eyes follow the motion of thesubject's head. However, a subject's eyes can swivel around the eye'scenter position. Thus the orientation of the eyes may change withrespect to the head orientation, and this should also be tracked. Theorientation of an eye may be measured with an eye-tracking system, whichwas discussed above.

In some embodiments, a head tracking system is specified by: headtracking rate, head tracking accuracy, and head tracking precision.Ideal head tracking specifications, which are not necessarily achievedin all embodiments, are: A head tracking rate that provides pose updatesat a rate that is imperceptible to the human eye. A head trackingaccuracy that provides head tracking pose updates that are accurate towithin a specified tolerance. And, a head tracking precision thatprovides head tracking precision that allows for corrections, which areunresolvable by an average human eye.

Head Tracking Rate

Head tracking is tracking a position of a subject's head. A HeadTracking Rate is a rate at which a head-tracking system can track themotion of a user's head (eyes). The minimum head-tracking rate for anHMD may be determined based upon the application. Human brains processdata at a maximum frequency of approximately 11 Hz. Therefore, in someembodiments, the projector renders each frame at a rate of about 30 Hz,as this frame update is imperceptible to the human eye. However, in someembodiments, pixels are refreshed by the projector at 60-120 Hz, inorder to remove amplitude flickering. It follows that in order to reduceperceptible position errors, some embodiments comprise a head-trackingrate that is specified no less than the project update rate (30 Hztypical), and no more than the refresh rate (120-240 Hz typical).

Head Tracking Accuracy

In some implementations, Head tracking not only tracks a user's headquickly (or at a rate greater than 30 Hz), it also tracks the user'shead to within a specified accuracy, for example to within 10 cm, 1 cm,or 0.1 cm depending on the application. Head tracking accuracy, 3905, isshown in FIG. 39B, as the distance, dPose_(a), 3905, between anestimated head pose, 3906, and an actual head pose, 3908. With respectto augmentation imagery, the head tracking accuracy can limit howwell-simulated content will appear placed in a scene. Head trackingaccuracy has a stronger effect on augmentations that are closer to auser than those at a relative distance because the position error leadsto an angular error given by:

$\begin{matrix}{{Pose}_{error} = \frac{dx}{z}} & (5)\end{matrix}$

Head Tracking Precision

Head Tracking Precision, 3910, refers to, how consistently the positionof the head is being tracked and is depicted in FIG. 39C. Even if thereis a nominal offset error that misaligns the real and virtualsimulations (inaccurate), it is possible to measure the same positionrepeatedly (precise). Head tracking precision may be given by the rateof error change as a function of time:

$\begin{matrix}{{\Delta \; {Pose}_{error}} = {\int_{0}^{\tau}{\frac{{dx}(t)}{z(t)}{dt}}}} & (6)\end{matrix}$

Where τ is an integration time dictated by the head-tracking rate.

In some embodiments, a head tracking-system comprises a motion capturesystem, as shown in FIG. 40. Typically, motion capture systems compriseone or more motion capture cameras, 4002, and one or more markers, 4004.Examples of motion capture systems include: Prime 41 from NaturalPointInc. DBA OptiTrack of Corvallis, Oreg.; Impulse X2 motion capture systemfrom PhaseSpace Inc. of San Leandro, Calif.; and Vicon Vantage of ViconMotion Systems Ltd. UK registered no. 1801446. An example of a stopmotion system specifically for head and face tracking is Vicon Cara ofVicon Motion Systems Ltd.

Ray Densification System

As explained above, anatomical retinal variations cause vision qualityto vary over a user's field of view. A ray densification system takesadvantage of anatomical retinal variations to increase the projector'sFOV. In some embodiments, this is accomplished by modifying the beamlaunchers to include ray densification optics. Some versions includepartially mirrored surfaces that segment projected imagery, into twopieces to be projected as: inner FOV, 4102, and outer FOV, 4104, asshown in FIG. 41.

Four different versions of ray densifying beam launchers are depicted inFIGS. 42A-D. FIG. 42A illustrates a pseudo-lantern beam launcher design.A collimated projector beam enters from the right and is split into twobeams by a 50/50 liquid crystal mirror (LCM), 4202. Half the lightpasses through the beam-splitter and reflects off a second LCM device,4204, on the left hand face of the beam launcher cube, opposite theentry face. The second LCM, has a central mirrored surface, 4206. Thislight is collimated by a lens, 4208, placed at the exit face of the beamlauncher cube. A second half of the light reflected from thebeam-splitter surface is retro-reflected back from a parabolic mirroredsurface, 4210. At the center of the parabolic surface, 4210, there is adark spot of low reflectivity, 4212.

In some embodiments, light passing through the pseudo-lantern beamlauncher will exit with a sharp FOV over the central 30 degree FOV, anda wide peripheral FOV, with less resolution. The rays have beencompacted so that they are denser in the central portion of the user'sFOV taking advantage of the anatomy of the human eye to produce animmersive wide FOV by putting optical information where it is mostneeded. The disadvantage of the pseudo-lantern beam launcher is itsrelatively poor throughput. Only 50% of the light entering the beamlauncher makes it out the output face of the device. However, the smallform factor this design allows makes it an attractive version of the raydensification system.

A second design, illustrated in FIG. 42B, is a 2-cube beam launcher. Inthis design, the inner and outer FOV are output coupled by 2 differentbeam launcher cubes. A first beam launcher cube, 4214, output couplesthe inner FOV. The rest of the beam passes through the beam splitter,4216, and enters a second beam launcher cube, 4218. This beam launcheroutput couples with a short focal length lens (or negative lens), 4220,to spread the outer FOV over a desired FOV of a user. In someembodiments, lenses on the output faces of the beam launcher cubes, 4208and 4220, comprise an adjustable focus optic, such as a liquid crystallens to provide adaptive control. The 2-cube beam launcher design hasthe obvious disadvantage that it greatly increases the size of the beamlauncher array.

A third design, illustrated in FIG. 42C, is a single diffractive opticalelement (DOE) beam cube. In an embodiment of this design, an entire beamis output coupled through a DOE, 4222, that spreads the beamnonlinearly. At the center, the inner FOV of the beam is modified by theDOE, 4222, as if by a normal collimating lens. And at the periphery, theouter FOV of the beam is modified by the DOE to be spread out over awide-angle. Embodiments of this version of a ray densification systemare small and have high light throughput. However, DOEs are typicallymore expensive to manufacture than conventional optics.

A fourth design, illustrated in FIG. 42D, is a DOE double beam launchercube. The DOE double beam launcher cube comprises two beam launchercubes. The inner FOV is output through a high-quality imaging lens,4208, and the outer FOV is output through a DOE wide-angle lens, 4224.Embodiments of the DOE double beam launcher cube benefit from use of theDOE wide-angle lens, 4224, and retain high image quality over the innerFOV.

Beam launcher cubes comprising ray densification optics may also beassembled into a BLA as described above. An embodiment of a BLAcomprising a two launcher cube, ray densification system is shown inFIG. 43A. An embodiment of a BLA comprising the pseudo-lantern beamlauncher is illustrated in FIG. 43B. An embodiment of a BLA comprisingthe DOE beam launcher cube is illustrated in FIG. 43C. As can be seen inFIGS. 43A-C, BLAs comprising single cube beam launchers are smaller thanBLAs comprising two cube beam launchers. However, there is littledifference in size between BLAs comprising the pseudo-lantern beamlaunchers and BLAs comprising the DOE cube launchers.

In some versions, light associated with augmentation imagery isprojected by a light generator over a wide field of view (FOV) of auser. The light is encoded by a field mapping, which comprises colorvalues and depth values associated with render data. In some versions,the field mapping comprises a higher resolution color mapping over aninner portion of light than an outer portion of light. The inner portionof light being imaged incident a central vision of the user. And, theouter portion of light being imaged incident a peripheral vision of theuser. In some version, the field mapping comprises a depth mappingcontaining focal depth values for accommodation of stereoscopicaugmentation imagery. The depth mapping can contain depth informationfor all light being projected. According to another embodiment, thedepth mapping contains focal depth values for the inner portion of lightthat is to be imaged at the central vision of the user, and provides nofocal depth data for the outer portion of light that is to be imaged atthe peripheral vision of the user. Some versions of the system, whichcomprise non-linear field mappings, further comprise: an eye-trackingsystem. The eye-tracking system provides eye-tracking data, in responseto a measured position of the user's pupil. A controller is used togenerate the field mapping in response to the eye-tracking data.

Head Mounted Display (HMD) Controller

In some embodiments, an HMD Controller is responsible for commanding allHMD sub-systems in the correct sequence to generate desired outputimagery. A version of an A-HMD Command process is depicted in FIG. 44.

Referring to FIG. 44, an HMD local processing, 4402, receives externalrender data, 4404, jitter sensor data, 4406, and eye-tracking data,4408. The HMD Local processing updates the HMD system state, and the HMDlocal processing then sends commands to the accommodating projector,4410, wide field projector, 4412, the jitter stabilization system, 4414,and the beam launching array, 4416. Exemplary data input rates of updatefor some embodiment systems are: 30 Hz for external render data rate,4418, 480 Hz for a jitter sensor data rate, 4420, and 480 Hz for aneye-tracking data rate, 4422. Exemplary accommodating projector ratesfor some embodiments are: 30 Hz for an accommodating projector framerate, 4424, 120 Hz for an accommodating projector refresh rate, 4426,and 480 Hz for an accommodating projector focus adjustment rate, 4428,projecting over four focal depths. Exemplary wide field projector ratesfor some embodiments are: 30 Hz for a wide field projector frame rate,4430, and 120 Hz for a wide field projector refresh rate, 4432. Anexemplary jitter stabilization refresh rate, 4434, for some embodimentsof a jitter stabilization system, 4414, is 120 Hz. And, an exemplarybeam launching array refresh rate, 4436, is 120 Hz.

A timing sequence undertaken by an example HMD controller is illustratedin FIG. 45. Referring to FIG. 45, there are three sequences of commandsthat are updated at three key commanding frequencies: a High-Rate, 4502,or about 480 Hz, a Mid-Rate, 4504, or about 120 Hz, and a Low-Rate,4506, or about 30 Hz:

Commands that are updated at the High-Rate comprise:

Receive Jitter Data, 4508, comprising: measured head motion jitter,

Receive Eye-Tracking Data, 4510, comprising: measured gaze vector; and

Update Accommodating Projector Focal Depth, 4512, comprising: refreshingthe accommodating projector's focal depth position.

Commands that are updated at the Mid-Rate comprise:

Update the Control States, 4514, comprising: update commands to thejitter stabilization system,

Command the beam launching array, 4516 comprising: updates to gazedirection of the BLA; and

Refresh Projector Systems, 4518, comprising: refreshing the projectorpixel for both the accommodating projector and the wide field projector.

Commands that are updated at the Low-Rate comprise:

Receive Render Data, 4520, comprising: un-rasterized augmentation data,

Update Accommodating Projector, 4522, comprising: RGB(x,y,z) projectorstate updates; and

Update the Wide Field Projector, 4524, comprising: RGB(x,y) projectorstate updates.

A-HMD Integrated System

An example configuration of an A-HMD is illustrated in FIG. 46. TheA-HMD shown in FIG. 46 comprises an accommodating projector, anaugmentation stabilization system, a ray densification system, and afirst portion of an image fusion system. Not shown in FIG. 46 is a beamcombiner that reflects output augmentation imagery from a beam launcharray incident an eye of a user.

Referring to FIG. 46, light, which is color encoded, is projected by aprojection source, 4602, and reflected by a MEMs mirror, 4604. The MEMsmirror scans the light in response to a field mapping. The fieldmapping, having been generated by a controller, defines augmentationimagery that is to be imaged within a field of view of a user. The lightbeing scanned upon an aperture stop, 4606, is imaged incident anadjustable focus optic, 4608. The adjustable focus optic, 4608, operatesin response to the field mapping in order to modify the vergence of thelight, thereby adding one or more focal depth values to the light. Insome embodiments the light is further acted upon by a lens, 4610. Thelight is then modified by an augmentation stabilization system. Theembodiment of the augmentation stabilization system shown in FIG. 46comprises a fast steering mirror (FSM), 4612, and a pupil adjustmentmirror (PAM), 4614, as well as a number of optics, 4616. The number ofoptics, within the augmentation stabilization system, image the light topupil planes, coincident with the FSM, 4612, and the PAM, 4614.Measurements relating to high frequency (greater than 10 Hz) changes toa pose of the user's head are made by a jitter sensor, 4618. The jittersensor, 4618, may comprise one or more electronic elements selected froma list of: accelerometers, gyroscopes, magnetometers and inertialmeasurement units (IMUs). FSM, 4612, and the PAM, 4614, translate thelight in response to small eye movement measurements correlated frommeasurement taken by the jitter sensor, 4618. Another lens, 4620, isused in some embodiments to reimage the light. The light in someembodiments is decreased in size by a compressor such that the diameterof the projected beam matches, or is smaller than an entrance aperture,4622, of the beam launching array (BLA), 4623. In the version an imagefusion system shown, the compressor comprises a first compressor lens,4624, and a second compressor lens, 4626. The light is input coupledinto the image fusion system and the beam launch array, 4623. Aneye-tracking system, 4628, measures a location of the user's pupil andthereby determines a gaze vector that is associated with the user's lineof sight. The light is output coupled from the beam launch array, 4623,from a beam launching cube, 4630. The beam launch cube, 4630, from whichthe light is reflected, is selected by a controller in response toeye-tracking data measured by the eye-tracking system, 4628, in order toultimately image the light at the eye of the user. Light being outputcoupled from the beam launching cube, 4630, and prior to being imaged atthe user's eye, is reflected by a curved beam combiner, not shown. Thecurved beam combiner being placed in the user's line of sight allows forthe light from the beam launch array to be reflected into the user'seye, in addition to external light from the surroundings. The abovedescribed series of events is duplicated for the user's second eyeallowing for stereoscopic perception of the augmentation imagery. Andthus, the user can see both the world, as it is around her, and beimmersed in augmentation imagery simultaneously. The HMD systems arecontrolled by an HMD controller, 4632, which causes the systems toupdate and refresh, at the above described rates. The user may move: hereyes, her head; and her body, without ever perceiving that some of whatshe is seeing, the augmentation imagery, is virtually generated, and nota palpable part of her surroundings, as her eyes would have her believe.

Additional Versions

A Liquid Crystal Wedge, or Wave Front Corrector, AugmentationStabilization System

As explained above, Augmentation jitter is the apparent motion ofaugmentation imagery relative to the user's view of the real world. Thiscan have the effect of making augmentation imagery shake or move inrelation the viewed scene. It is a problem for AR head mounted displays(HMDs) because it breaks a user's sense of immersion. The motion of theHMD relative to the world-view creates projected augmentation jitter.The motion of a human head relative to the real world may be representedby 6 degrees of freedom: x,y,z, x_rot, y_rot, z_rot. If we assume smallrotations and translations of the head position, then the projectedaugmentation imagery is just translated relative to the world-view.According to the Fourier shift theorem, a shift in the image planeresults in a phase gradient in the pupil plane (tip/tilt) and viceversa. Therefore, translations of augmentation imagery relative to theworld-view may be corrected by applying a tip/tilt to the exit pupil ofthe projector system. Normally this is achieved with a tip/tilt mirror(jitter mirror), such as a MEMS mirror. However, the same function maybe achieved with a set of liquid crystal (LC) wedges.

FIGS. 47A-B depicts a wave front corrector or liquid crystal wedge,4702, introducing a tip gradient, 4704. An input beam, 4706, is shownentering the wave front corrector perpendicularly from the top. The wavefront corrector is constructed as described above. A liquid crystallayer, 4708, is located between two transparent electrodes, 4710 and4712, and two optically clear substrates, 4714. In the shown embodiment,a common electrode, 4710, is un-patterned and electrically connected toground. A patterned electrode, 4712, is arranged, such that electrodesare patterned in individual strips. It can be seen from FIG. 47A that apotential V1 is introduced on a left most region of the patternedelectrode, 4712. The nine regions to the right of the left most regionhave a potential applied to them, which is lower than the potential inthe region to their immediate left. The right most region of thepatterned electrode has little-to-no potential, V10, applied to it. Thepotentials V1 through V10 are applied in a steady ramp from a high at V1to a low at V10. The applied voltage results in a change in anorientation of liquid crystal molecules within the liquid crystal layer,4708. The change in orientation of liquid crystal molecules causes achange in an index of refraction across the surface of the wave frontcorrector, thereby introducing a tip to the output beam, 4716.

FIG. 48 depicts a number of wave front correctors, such as liquidcrystal wedges (LCW), which are patterned to produce a tip/tilt gradientin a passing beam. A LCW has an electrode pattern comprising: a seriesof strips through which a non-uniform potential is applied across theliquid crystal layer. A phase gradient (tip/tilt) may be introduced byvarying the potential in a linear ramp, causing the input beam todeflect at an angle equal to an induced wedge angle. A horizontal wedge,4802, creates a horizontal tilt, 4804, in the horizontal plane andsteers the beam to the left or right. A vertical wedge, 4806, creates atilt in the vertical plane, 4808, and steers the beam up or down. Thevertical wedge, 4806, and the horizontal wedge, 4802, acting in concertproduce a total tilt, 4810, in two planes. Some embodiments of anaugmentation stabilization control system comprise: at least two liquidcrystal wedges, and a jitter sensor (e.g. a gyro, accelerometer).Alternatively, a single wave front corrector may be configured withmultiple etched electrode layers, in order to produce both tip and tiltgradients.

Augmentation Stabilization Control System to Correct Optical Jitter

An augmentation stabilization control system may be operated in eitheropen, 4902, or closed loop, 4916. The two control processes areillustrated in FIG. 49. Optical jitter as described in detail above,results from HMD movement relative a user's eyes. In the open-loop mode,4902, the projector's optical system is held rigidly in respect to theuser, and therefore no feedback is required. The open loop augmentationstabilization system then operates to correct augmentation jitter, andnot optical jitter. As described in detail above, a jitter sensor, 4904,provides data to a jitter controller, 4906, which operates a X-Tilt LCWedge, 4908, and a Y-Tilt LC Wedge, 4910, to adjust incoming opticaldata, 4912, and reposition the resulting image, 4914. The augmentationstabilization control system may also be operated in closed-loop mode,4916, if the projector system is not held rigidly. The closed-loopcontrol system can correct for optical projector jitter by applying ajitter correction to offset the error. In this case, the feedback, 4918,is a measurement of the projector jitter directly, either through inscene motion detection, or an optical jitter sensor in the projectorbeam's path. Examples of jitter sensors that detect optical jitterinclude: shearing interferometer, Shack-Hartmann sensor, pyramid sensor,curvature sensor, and speckle jitter sensor. Regardless of jittersensing means, the sensed jitter of the projector relative to itsinitial axis may be corrected for, with a closed-loop augmentationstabilization control system. The examples of open and closed loopaugmentation stabilization control systems explained in reference toFIG. 49 comprise liquid crystal wedges. Other augmentation stabilizationcontrol systems, such as those described in detail above that comprisefast steering mirrors, may also be operated in closed loop mode in orderto reduce optical jitter.

Alternative Embodiment: Augmentation Stabilization Control System toCorrect Jitter with an Embodiment Comprising a Waveguide

In an alternative embodiment, projected light, 5002, associated withdisplay imagery is projected through a waveguide, 5004. FIG. 50illustrates a stabilization control system configured to correct jitter,5006. Jitter causes projected light, 5002, to be dislocated relative auser's eye, 5008, as it is being emitted from the waveguide, 5004. Afirst wave front corrector, 5010, is located between the waveguide,5004, and the eye, 5008. In some embodiments, the wave front correctoris configured to produce a tip and/or a tilt. As described above, thetip/tilt introduced by the first wave front corrector, 5010, cantranslate the projected light, and the associated augmentation imagery,thus correcting the jitter. In some embodiments, the projected light,5002, and the light from the surroundings, 5012, have the samepolarization, or both light sources are unpolarized. In this case, thefirst wavefront corrector, in correcting the mislocation of theprojected light, 5002, also mislocates the light from the surroundings,5012. A second wave front corrector, 5014, is used to produce a contrarytip and/or tilt, thus acting to undo the effects of the first wave frontcorrector on the light from the surroundings, 5012. In alternativeembodiments, only the first wave front corrector is required, becausethe light from the surroundings is of a different polarization than theprojected light or is unpolarized. The augmentation stabilizationcontrol system further comprises a jitter sensor, described above, whichmeasures augmentation and/or optical jitter.

Multi-Projector Systems

In some embodiments, an A-HMD comprises a light generator having morethan one projector to generate augmentation imagery to project into asingle eye of a user. For example, a ray densification system maycomprise two projectors, as shown in FIG. 51. A first projector system,5102, comprises an accommodating projector system, 5103, that renders aninner FOV. In some versions, the first projector system renders only theinner 30° (+/−15°) of the total FOV by applying an inner FOV aperturestop. A second projector system, 5104, comprises a wide fieldnon-accommodating projector, 5105, that renders an outer FOV by applyingthe outer FOV aperture stop. In some versions, the second projector isnon-accommodating, because the human eye cannot see in focus beyond theinner 30° FOV (+/−15°). Even though a user can have a sense of depth dueto stereoscopic vision, the image quality is poor and symbols are notrecognizable. Beams from the first and second projector systems arebrought back together in a pupil plane and recombined by a beamcombiner, 5106. This recombined pupil may then be re-imaged onto theentrance pupil of the eye (or onto another intermediate pupil) and theimage that forms on the retina of the eye will merge the inner and outerFOV images to form a total FOV image. By doing this we may increaseeither the refresh rate or the resolution of the rendered scene ascompared to a projector that renders the wide FOV. The projector of theinner FOV is an accommodating projector, because the human eye canaccommodate for focal depth over the central portion of the FOV.However, the human eye can focus on only one depth at a time, so if theeye tracking precision is high enough only one focal depth need berendered. Eye-Tracking precision specifications may be relaxed somewhatif the projector is run faster and includes multiple “nearby” focaldepths. In some embodiments, both inner, 5102, and outer, 5104,projection systems comprise augmentation stabilization systems, 5108.Other embodiments, comprise a single augmentation stabilization systemthat acts on both the inner and outer FOV of the displayed image.

Eye Refractive Error

The human eye's ability to accurately form an image of an object variesfrom person to person and is measured by the eye's refraction error(also known as wave front error). An eye's refraction error may bemeasured in several ways: an autorefractor, a retinascope, or with awave-front sensor. Wave front error is a two-dimensional measurement ofthe wave front's departure from an ideal planar wave front. The 2Dpatterns are often decomposed into Zernike modes, because the modes mapwell into optical distortions such as defocus, astigmatism, and coma.Zernike modes are shown in FIG. 52, grouped according to their symmetrynumber, M In general, the more diversity of information that isretrieved, the more accurately the refraction error may be decomposedinto Zernike modes. Often times very little diversity is used and thetotal refractive error integrated over all modes is used as a metric ofeye performance.

Autofocus

Head Mounted Displays (HMDs) can be configured to fit a variety ofdifferent people with different shaped heads, and different opticalprescriptions. The distance between a projector and the pupil of auser's eye (the HMD eye depth) will vary from person to person.Therefore, some embodiments of an accommodating HMD adjust theprescription of the system to compensate for variable HMD eye depth.

Some embodiments of an autofocus optical system use a control systemconsisting of a sensor and a tunable optical element to focusautomatically. Auto-focus methods are, active, passive, or a hybrid ofthe two. An active autofocus system measures the distance to an objectby emitting a probe signal and measuring the probe's response. Whereas apassive system, determine correct focus by measuring light from anobject itself without an active probe.

Examples of active probes include: ultrasonic sound waves, radio waves,or infrared light pulses. In either of these cases, measuring the pulsefrequency Doppler shift or the pulse's time of flight is used todetermine the range to an object. Examples of passive autofocus systemsinclude: phase detection systems and contrast detection systems. In lowlight, hybrid systems are often used that use an assist lamp toilluminate an object so that a passive detection technique is viable.

Some versions of an autofocus system comprise a liquid crystal (LC) lensas the tunable optical element. In this case, the focal length of the LClens is varied until a sharp image forms in a predetermined test plane(possibly the retina of the eye). An exemplary system uses two lensescombined to form an effective lens that will exactly image the projectedlight onto the retina of the user's eye. The effective focal length ofan optical system, f_(eff), composed of two lenses is given by:

$\begin{matrix}{\frac{1}{f_{eff}} = {\frac{1}{f_{1}} + \frac{1}{f_{2}} - \frac{d}{f_{1}f_{2}}}} & (7)\end{matrix}$

where f₁ and f₂ are the focal lengths of the two lenses and d is thedistance between them. The combined lens system will form an image at adistance, s_(i), behind the combined lens when an object is placed adistance, s_(o), in front of the combined lens as given by:

$\begin{matrix}{\frac{1}{f_{eff}} = {\frac{1}{s_{o}} + \frac{1}{s_{i}}}} & (8)\end{matrix}$

A typical autofocus system would adjust the distance d, between thelenses to adjust the location where an image is formed, s_(i), byadjusting the effective focal length, f_(eff). However, with an LC lenswe can directly adjust the focal length of one or both of the lenses inthe combined lens system. This allows you to adjust the effective focallength, f_(eff), without the need to physically move a lens.

As a control system, the first lens is the lens of a user's eye withfocal length, f₁ and the second lens is an adjustable LC lens withvariable focal length, f₂. The focal length of the LC lens is adjusteduntil it is matched correctly with f₁ to form a combined lens with thecorrect effective focal length, f_(eff), for a given object distance,s_(o), and image distance, s_(i), where s_(i) is specific to a user'seye. The LC autofocus system can correct for both the variation in eyeprescription as well as the variable distance between the combined lenssystem and the user's eye.

For an afocal system, there is no net curvature of the wavefront. Inother words, a collimated beam entering the entrance pupil is reimagedat the exit pupil of the system. In this case the effective focal lengthis infinity (object is at infinity). This is achieved for a two-lenssystem by separating the lenses by a distance equal to the sum of theirfocal lengths:

d=f ₁ +f ₂   (9)

This is a simple case of an autocollimator, where now we adjust thefocal length, f₂, of the LC lens, until it matches d-f₁, which occurswhen the exit pupil wave front is planar. In this case the combined lenssystem will form the best image on the back retina of the user's eye,which can be used as the control sensor.

With user feedback, best focus may be found by adjusting the focallength of the LC Lens until the user indicates that best focus has beenreached. Otherwise, 1D (diodes and arrays of diodes) or 2D (camerasystems: CCDs, EMCCDs, etc.) optical sensor may be used in anautoreflector configuration as shown in FIG. 53.

Referring to an exemplary autoreflector shown in FIG. 53, a lightsource, 5302, produces light, which is collimated by a collimating lens,5304. The light reflected from the back of the retina, 5306, will beimaged by a combined lens system composed of the user's eye, f₁, 5308,and an LC lens, f₂, 5310. A beam splitter, 5312, is used as a beamreturn mirror. A second detector, 5314, is placed off-axis with theuser's eye. The combined lens will image the retina some distance behindthe combined lens system. Now f₂ is adjusted until the combined lenssystem maximizes the sharpness or contrast metric in the image plane ofthe autofocus sensor system.

Low-Order Wave-Front Control with Liquid Crystal Optics

An embodiment of an Autofocus system as described above also adjusts aprescription of the system to account for the optical prescription of auser's eyes. This could be useful if the HMD is operated in a pure VRmode, where the system could be used to correct for a user'sprescription so she doesn't need to use her own eyewear. In an AR mode,the LC control system can be pared with a second system that correctsthe external “see-through” optical system.

For low order correction, we correct more Zernike modes than just focus.Instead we correct the low order, 5402, Zernike modes that dominate therefractive error of the user's eye, typically this is the first nineZernike modes. This is achieved with a stack of LC devices similar tothe LC lens. Each element of the stack has an anode with thedistribution pattern of one of the low order Zernike modes (see FIG.54). A stack of 8 may be achieved in less than a centimeter wide opticalstack. Such a device may be used to offset the low-order error of auser's eye, to achieve best image quality. This effectively corrects forthe prescription of a user's eye. FIG. 54 depicts what Zernike modes arecontrolled for each of three wavefront control systems. The First systemjust controls jitter (tip/tilt) and focus (autofocus), 5404. This is thecurrent system design. However, with the LC stack Zernike corrector, alow-Order control system may be obtained. If even more Zernike modes areaccounted for, a higher-Order control, 5406, will control more modesthan just the first nine modes.

Virtual Reality HMD

Above a foveated optical system is described as being incorporated in anaugmented reality (AR) head mounted display (HMD). Another, suitableapplication for a foveated optical system, or a wave front corrector, isvirtual reality (VR) HMDs. VR HMDs do not pass visible light from thesurroundings to a user. Instead, the user is fully immersed in thevirtual imagery provided by the VR HMD. Currently available stereoscopicVR HMDs, such as the Oculus Rift, do not accommodate the stereoscopicvirtual imagery. That is to say: the user is provided with a dynamicprecept of depth through parallax between the stereoscopic images, and astatic focal depth cue through the displayed light. For VR HMDs thefocal depth is typically held constant at optical infinity. It ishypothesized that the focus of a person's eyes is interrelated withtheir precepts of depth from stereoscopic viewing. Dysfunction thereforearises when the stereoscopic view tells the user's mind that the imageis at a certain depth, but the user's eyes must focus at a differentdepth in order to bring the image in to focus. For some users, this canresult in poor virtual reality experiences, and even discomfort, whichcan persist after the user has removed the HMD. A wave front correctormay be used to alter the Zernike mode associated with focus, in order tochange the focal depth of light projected in a VR HMD.

One half of an embodiment of a VR HMD, 5501, incorporating a wave frontcorrector is illustrated in FIG. 55. In some embodiments a VR HMD isconfigured with an OLED display, 5502. Alternatively, a liquid crystaldisplay (LCD), or similar may be used. OLED displays are in many wayspreferential, as they do not require backlighting, and can therefore:achieve higher contrast ratios, and be made thinner and lighter thanLCDs. As shown in FIG. 55, a wave front corrector, 5504, is placedbetween the display, 5502, and a user's eye, 5506. The wave frontcorrector may be configured to alter a focal depth of the virtualimagery, by altering the focus of a wave front, as described above. Asdescribed above some embodiments of a wave front corrector for VRapplications comprise: patterned, 5508, floating, 5510, and groundelectrodes, 5512; conductors, 5513; one or more substrates, 5514; aSilicon Oxide layer, or similar, 5516; one or more alignment layers,5518; and a liquid crystal layer, 5520.

In other embodiments, the VR HMD further comprises an eye-trackingsystem, and the wave front corrector is operationally responsive to eyeposition, such that a region of interest, 5604, coincides with a viewingangle of the user. As described above, the wave front corrector may beconfigured for foveated operation and correct the region of interest,5604, such that it is at a higher resolution than the scene outside theregion of interest, 5606. FIG. 56 illustrates a side-by-side view ofvirtual imagery as displayed by a VR HMD configured with a foveatedoptical system. While using the VR HMD the user is unable to view hersurroundings. Instead, the user may view only the scene displayed by theVR HMD. In the embodiment shown in the FIG. 56, an eye tracking system,not shown, measures eye positions of the user's eyes, and the wave frontcorrector corrects aberrations in a region of interest, 5604, which isgenerally located where the user looking. In another embodiment, one ormore focal depths may be applied within the region of interest, by usinga variable focus optic, or a wave front corrector that can exciteZernike mode 4, or defocus. In some embodiments, the resolution withinthe region of interest is diffraction limited, or limited by theresolution of the display, 5502. In some embodiments, a wave frontcorrector is configured in a VR HMD, in place of conventional optics.Some VR HMDs, such as those similar to Google Cardboard, use a single 25mm biconvex lens with a 40 mm focal length for each eye. A 25.4 mm N-BK7biconvex lens having a 40 mm focal length (Thorlabs Part No. LB1027-A)has a center thickness of greater than 6 mm, and will produces imageswith spherical aberrations. As described above a wave front correctormay be constructed which is less than 1 mm thick, and may be configuredto provide diffraction limited imaging.

Unmanned Aircraft

Unmanned aircraft, or drones, 5701, are increasingly common, and areused in various applications. Typically, drones are outfitted with acamera, 5703, such that the operator may take pictures, videos, oroperate the drone from the drone's vantage point. Much like therequirements for HMDs, a drone imaging system provides a wide field ofview, such that the drones surroundings may be perceived by the viewer.Many drones have camera's that provide a field of view greater than70°-90°, however the “fast” optical systems required to provide thesewide angles, typically produce more aberrations. Therefore, the imagesproduced over a wide field of view may be of poor quality.Alternatively, conventional wide angle optical systems include manyelements in order to correct the many aberrations. Conventional wideangle optical systems are therefore large and heavy, to descriptorswhich are typically unwelcome payload on small aircraft. Although a widefield of view may be needed to operate the drone, or understand thedrone's vantage point, there is often a region of interest, within thelarger field of view, which is the focus of the viewer. It isadvantageous for a drone imaging system that allows a wide field of viewand a high quality region of interest within the field of view, which isgenerally free from aberrations.

Unmanned vehicles have found applications in military and civilianmarkets. FIG. 57 illustrates two different types of unmanned aircraftthat include at least one camera. FIG. 57A illustrates a fixed wing typeunmanned airplane. FIG. 57B shows a quadcopter unmanned aircraft.

FIG. 58 illustrates the view from a camera configured with a wide-anglefoveated optical system in a drone application. It can be appreciatedfrom FIG. 58 that the wide-angle foveated optical system allows for awide-angle view of the surroundings, 5802. Additionally, a region ofinterest (ROI), 5804, within the field of view has been corrected and isfree of aberrations. This is advantageous for many unmanned aircraftapplications. For example, in FIG. 58 a small bird can be seen withinthe ROI, 5804, even as a wide-angle is displayed. This allows, a birdwatching enthusiast to determine the species of the bird, while seeingenough of the surroundings to safely operate the drone.

Although the present invention has been disclosed in considerable detailwith reference to certain preferred versions thereof, other versions arepossible. For example, in an alternative embodiment, multiple wave frontcorrectors are configured to correct and adjust different modes. A firstwave front corrector may be configured only to correct astigmatic modes(described above), and second wave front corrector may be positionedadjacent the first wave front corrector. In some embodiments, the secondwave front corrector corrects other types of aberrations. Additionally,the second wave front corrector is configured to alter the focus of thepassing light, and/or introduce tip/tilt gradients to correct forjitter, according to focus and jitter control systems as describedabove.

Another alternative embodiment is illustrated in FIG. 59. In this case,the number of etched electrode layers present in the wave frontcorrector may be reduced. For example, corrections of astigmaticaberrations, over many field angles typically require at least twoelectrode patterns, see above. Instead, a foveated imaging system, 5900,having a selectably rotating wave front corrector, 5902, allows forastigmatic aberrations to be corrected around all field angles with onlyone electrode pattern. In some embodiments, rotation of the wave frontcorrector is achieved through a system comprising: bearings, a magnet ona wave front corrector mount, and an electro magnet; the system beingconfigured such that energizing the electro-magnet rotates the wavefront corrector a known amount. Alternatively, rotation of the wavefront corrector is achieved by a piezo actuator, a cog and motor, or agalvanometer. In some embodiments the foveated optical system, 5900,also includes two conventional non-adaptive optics, 5904 and 5906.

Yet another alternative embodiment comprises multiple electrode patternson a single electrode layer. The more complicated single-layer electrodepatterns have sufficient electrodes to produce wave front changes inmultiple Zernike modes. These single-layer electrode patterns requiremore complex conductor routing, but reduce the total number of electrodelayers required in the LC cell. In other embodiments, a floating layeris included that includes patterns that affect the wave frontcorrector's excitation of multiple wave front error sources, such as:focus, vertical astigmatism, and oblique astigmatism.

Certain control elements of the subject matter described herein can beimplemented in digital electronic circuitry, or in computer software,firmware, or hardware, including the structural means disclosed in thisspecification and structural equivalents thereof, or in combinations ofthem. Such subject matter described herein can be implemented as one ormore computer program products, such as one or more computer programstangibly embodied in an information carrier (e.g., in a machine readablestorage device), or embodied in a propagated signal, for execution by,or to control the operation of, data processing apparatus (e.g., aprogrammable processor, a computer, or multiple computers). A computerprogram (also known as a program, software, software application, orcode) can be written in any form of programming language, includingcompiled or interpreted languages, and it can be deployed in any form,including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment. Acomputer program does not necessarily correspond to a file. A programcan be stored in a portion of a file that holds other programs or data,in a single file dedicated to the program in question, or in multiplecoordinated files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program can be deployed to beexecuted on one computer or on multiple computers at one site ordistributed across multiple sites and interconnected by a communicationnetwork.

The processes and logic flows described in this specification, includingthe method steps of the subject matter described herein, can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions of the subject matter describedherein by operating on input data and generating output. The processesand logic flows can also be performed by, and apparatus of the subjectmatter described herein can be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processor of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for executing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. Information carrierssuitable for embodying computer program instructions and data includeall forms of nonvolatile memory, including by way of examplesemiconductor memory devices, (e.g., EPROM, EEPROM, and flash memorydevices); magnetic disks, (e.g., internal hard disks or removabledisks); magneto optical disks; and optical disks (e.g., CD and DVDdisks). The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

The subject matter described herein can be implemented in a computingsystem that includes a back end component (e.g., a data server), amiddleware component (e.g., an application server), or a front endcomponent (e.g., a client computer having a graphical user interfacethrough which a user can interact with an implementation of the subjectmatter described herein), or any combination of such back end,middleware, and front end components. The components of the system canbe interconnected by any form or medium of digital data communication,e.g., a communication network. Examples of communication networksinclude a local area network (“LAN”) and a wide area network (“WAN”),e.g., the Internet.

Throughout this disclosure various functions for an athletic headmounted display have been specified in terms of human perceptibilityrather than quantifiable units. It is the intention of the author thatthese specifications, given in these terms, to be as instructive as ispossible to the reader wishing to practice that which is disclosed. Thespecifications as defined in terms of perceptibility are, at times inthe disclosure, defined as best case or ideal, and should not be viewedas limiting. It should be noted that in some embodiments, within thescope of the present disclosure; the presence of engineering trade-offsand design constraints require the function of a system or subsystem tobe outside the functional specifications as described above.

Although the present invention has been disclosed in considerable detailwith reference to certain preferred versions thereof, other versions arepossible. For example: discrete beam launching elements that are notcubic in shape. More than two projector's may be used to display imagesover a wide field of view having different image resolutions deliveredto different regions of a user's field of view with differentcapabilities to resolve. A beam launching device may be implementedwithout a beam splitter, and instead be positioned to reflect imagerydirectly into a user's eye. Additionally, the following is a list ofembodiments contemplated by the present disclosure:

A1. A system for selectively reflecting an image, the system comprising:

a substrate having a first surface through which light is input coupled;and

a switchable reflector located within the substrate and oriented betweenthe first surface and a second surface of the substrate; the switchablereflector being configured to selectively reflect the light through athird surface of the substrate.

A2. The system of embodiment A1, wherein the light is visible light

A3. The system of embodiment A1, wherein the switchable reflector isswitchable between reflective and non-reflective states at a rate atleast as great as 30 Hz.

A4. The system of embodiment A3, wherein the switchable reflector isswitchable between reflective and non-reflective states at a rate nogreater than 120 Hz.

A5. The system of embodiment A1, wherein the light has a width at leastas great as 1 mm.

A6. The system of embodiment A5, wherein the switchable reflectorcomprises a clear aperture greater than 1 mm.

A7. The system of embodiment A1, wherein the light is reflected incidenta pupil of a user.

A8. The system of embodiment A7, wherein the light is reflected incidenta retina of the user.

A9. The system of embodiment A8, wherein the light is reflected incidentthe retina of the user, such that the light is imaged within a centralfield of view of the user.

A10. The system of embodiment A1, wherein the light comprisesaugmentation imagery.

A11. A system for selectively reflecting an image, the systemcomprising:

a substrate that is optically transparent to light being input coupledinto the substrate through a first surface,

a switchable reflector located within the substrate and oriented betweenthe first surface and a second surface of the substrate; the switchablereflector being configured to selectively reflect the light through athird surface of the substrate; and

an adjustable focus optic, such that a wave front of the light may beselectively shaped.

A12. The system of embodiment A11, wherein the adjustable focus opticcomprises a liquid crystal lens.

A13. A system for selectively reflecting an image, the systemcomprising:

a substrate that is optically transparent to light,

an adjustable gradient optic, such that the light may be selectivelysteered,

a first surface of the substrate through which the light may be inputcoupled; and

a switchable reflector located within the substrate and oriented betweenthe first surface and a second surface of the substrate; the switchablereflector being configured to selectively reflect the light through athird surface of the substrate.

A14. The system of embodiment A13, wherein the adjustable gradient opticcomprises a liquid crystal wedge.

A15. A system for selectively reflecting an image, the systemcomprising:

a substrate that is optically transparent to light,

an adjustable gradient optic, such that the light may be selectivelysteered,

a first surface of the substrate through which the light may be inputcoupled,

a switchable reflector located within the substrate and oriented betweenthe first surface and a second surface of the substrate; the switchablereflector being configured to selectively reflect the light through athird surface of the substrate.; and

an adjustable focus optic, such that a wave front of the light may beselectively shaped.

A16. A system for selectively reflecting a light from a plurality ofswitchable reflectors, the system comprising:

an array, comprising a plurality of switchable reflectors, and having alight directed incident and substantially along the array,

the array being configured to selectively reflect the light from one ormore of the switchable reflectors.

A17. The system of embodiment A16, wherein the array is curved.

A18. The system of embodiment A16, wherein the light is visible light

A19. The system of embodiment A16, wherein the plurality of switchablereflectors are switchable between reflective and non-reflective statesat a rate at least as great as 30 Hz.

A20. The system of embodiment A19, wherein the plurality of switchablereflectors are switchable between reflective and non-reflective statesat a rate no greater than 120 Hz.

A21. The system of embodiment A16, wherein the light has a width atleast as great as 1 mm.

A22. The system of embodiment A21, wherein the plurality of switchablereflectors comprises at least one clear aperture greater than 1 mm.

A23. The system of embodiment A16, wherein the light is reflectedincident a pupil of a user.

A24. The system of embodiment A23, wherein the light is reflectedincident a retina of the user.

A25. The system of embodiment A24, wherein the light is reflectedincident the retina of the user, such that the light is imaged within acentral field of view of the user.

A26. The system of embodiment A16, wherein the light comprisesaugmentation imagery.

A27. A system for selectively reflecting a light incident a user'spupil, the system comprising:

an eye tracking system that generates eye-tracking data related to aposition of a user's pupil; and

an array, comprising a plurality of switchable reflectors, and having alight directed incident and substantially along the array,

the array being configured to selectively reflect the light from one ormore of the switchable reflectors, in response to the eye-tracking data.

A28. A system for selectively reflecting a light from a plurality ofswitchable reflectors, the system comprising:

an array, comprising a plurality of switchable reflectors, and having alight directed incident and substantially along the array,

the array being configured to selectively reflect the light from one ormore of the switchable reflectors; and

a beam combiner oriented such that the light reflected from the array isdirected incident the beam combiner.

A29. A system for selectively reflecting a light from a plurality ofswitchable reflectors, the system comprising:

an array, comprising a multitude of switchable reflectors, and having alight directed toward and substantially along the array,

The array being configured to selectively reflect the light from one ormore of the switchable reflectors; and

an adjustable gradient optic incident the light, configured toselectively steer the light.

A30. A system for selectively projecting light from a multitude oflocations, the system comprising:

an array, comprising a multitude of switchable reflectors, and having alight directed toward and substantially along the array,

The array being configured to selectively reflect the light from one ormore of the switchable reflectors; and

an adjustable focus optic incident the light, configured to selectivelyshape a wave front of the light.

A31. An accommodating projection system comprising:

a field mapping comprising: a color mapping, and a depth mapping,

a light generator that projects light, in response to the color mapping;and

an adjustable focus optic that selectively shapes a wave front of thelight in response to the depth mapping.

A32. The system of embodiment A31, wherein the adjustable focus opticcomprises an annular spatial light modulator.

A33. The system of embodiment A31, wherein the adjustable focus opticcomprises a liquid crystal lens.

A34. The system of embodiment A31, wherein the light comprisesaugmentation imagery.

A35. An accommodating projection system comprising:

a controller that generates a color mapping, and a depth mapping,

a light generator that projects light, in response to the color mapping;and

an adjustable focus optic that selectively shapes a wave front of thelight in response to the depth mapping.

A36. An accommodating projection system comprising:

an eye tracking system that generates eye-tracking data, in response toa position of a user's pupil.

a controller that generates a color mapping, and a depth mapping inresponse to the eye-tracking data,

a light generator that projects light, in response to the color mapping;and

an adjustable focus optic that selectively shapes a wave front of thelight, in response to the depth mapping.

A37. An accommodating projection system comprising:

an eye tracking system that generates eye-tracking data, having a gazevector, in response to a position of a user's pupil.

a controller that generates a color mapping, and a depth mapping inresponse to the eye-tracking data; wherein the depth mapping comprisesat least one depth value associated with a distance along the gazevector,

a light generator that projects light, in response to the color mapping;and

an adjustable focus optic that selectively shapes a wave front of thelight, in response to the at least one depth value.

A38. The system of embodiment A37, further comprising:

a calculated depth value associated with a distance along the gazevector; and

at least one associated depth being proximal the calculated depth value.

A39. An accommodating projection system comprising:

a controller that sequentially generates color mappings, and depthmappings,

a light generator that projects light, in response to a color mapping;and

an adjustable focus optic that selectively shapes a wave front of thelight, in response to a depth mapping.

A40. An accommodating projection system comprising:

an eye tracking system that generates eye-tracking data, in response toa position of a user's pupil.

a controller that sequentially generates color mappings, and depthmappings in response to the eye-tracking data,

a light generator that projects light, in response to a color mapping;and

an adjustable focus optic that selectively shapes a wave front of thelight, in response to a depth mapping.

A41. An accommodating projection system comprising:

an eye tracking system that generates eye-tracking data having a gazevector, in response to a position of a user's pupil.

a controller that sequentially generates color mappings, and depthmappings in response to the eye-tracking data; wherein a depth mappingcomprises at least one depth value associated with a distance along thegaze vector,

a light generator that projects light, in response to a color mapping;and

an adjustable focus optic that selectively shapes a wave front of thelight, in response to the at least one depth value.

A42. An accommodating projection system comprising:

a controller that sequentially generates color mappings at a projectorupdate rate,

a light generator that projects light, at a refresh rate, in response toa color mapping; and

an adjustable focus optic that selectively shapes a wave front of thelight, at an accommodation update rate, in response to a depth mapping.

A43. The system of embodiment A42, wherein the accommodation update rateis at least 30 Hz.

A44. The system of embodiment A42, wherein the accommodation update rateis at least as great as the refresh rate.

A45. The system of embodiment A42, wherein the projector update rate isno greater than 240 Hz.

A46. A system for generating eye-tracking data and determining a gazevector of a user's eye, the system comprising:

a light source that generates a diffuse light, being at least partiallydirected toward a user's eye,

a camera located proximate the user's eye, such that light reflectedfrom the user's eye is measured by the camera and a pupil of the user'seye may be determined; and

a centroiding system that algorithmically determines eye-tracking data,in response to the pupil and a known orientation of the camera withrespect to the user's eye.

A47. A system for generating eye-tracking data and determining a gazevector of a user's eye, the system comprising:

a light source that generates a diffuse light, being at least partiallydirected toward a user's eye,

a camera located proximate the user's eye, such that light reflectedfrom the user's eye is measured by the camera and a pupil of the user'seye may be determined; and

a centroiding system that algorithmically determines eye-tracking data,in response to the pupil and a known orientation of the camera withrespect to the user's eye; wherein the eye-tracking data comprises agaze vector.

A48. A system for sensing, correcting and updating a location ofaugmentation imagery, in response to high frequency movements, thesystem comprising:

a light generator that projects light incident a user's pupil,

a jitter sensor that generates jitter data; and

an augmentation stabilization control system that adjusts a position ofthe light at the user's pupil, in response to the jitter data, at anaugmentation stabilization rate.

A49. The system of embodiment A48, wherein the augmentationstabilization rate is at least 30 Hz.

A50. The system of embodiment A48, wherein the light at the user's pupilcomprises augmentation imagery.

A51. The system of embodiment A48, wherein the jitter sensor comprisesand inertial measurement unit.

A52. The system of embodiment A48, wherein the augmentationstabilization control system comprises an adjustable gradient optic.

A53. The system of embodiment A52, wherein the adjustable gradient opticcomprises a liquid crystal wedge.

A54. The system of embodiment A48, further comprising a projectionjitter sensor.

A55. The system of embodiment A48, further comprising a feedbackmeasured in response to a position of the light.

A56. The system of embodiment A48, wherein the jitter data is related tomovements of a user's eye.

A57. The system of embodiment A48, wherein the jitter data is related tomovements of a user's head.

A58. The system of embodiment A48, wherein the jitter data is related tomovements of the light.

A59. A system for sensing a location of augmentation imagery, inresponse to high frequency movements, the system comprising:

a controller that generates a field mapping at a projector update rate,

a light generator that projects light associated with the field mappingat a refresh rate at least as great as the projector update rate; and

a jitter sensor that generates jitter data at a jitter sensing rate atleast as great as the projector update rate.

A60. A system for sensing a location of augmentation imagery in responseto high frequency movements, the system comprising:

a controller that generates a field mapping at a projector update rate,

a light generator that projects light associated with the field mappingat a refresh rate at least as great as the projector update rate,

an eye tracking system that generates eye-tracking data in response to aposition of a user's pupil at an eye-tracking rate at least as great asthe projector update rate; and

a jitter sensor that generates jitter data at a jitter sensing rate atleast as great as the projector update rate.

A61. A system for sensing, correcting and updating the location ofaugmentation imagery, in response to high frequency movements, thesystem comprising:

a controller that generates a field mapping at a projector update rate,

a light generator that projects light associated with the field mapping,

a jitter sensor that generates jitter data, in response to movements;and

an augmentation stabilization control system that adjusts the positionof the light at the user's pupil, in response to the jitter data, and atan augmentation stabilization rate that is at least as great as theprojector update rate.

A62. A system for sensing, correcting and updating the location ofaugmentation imagery, in response to high frequency movements, thesystem comprising:

a controller that generates a field mapping at a projector update rate,

a light generator that projects light associated with the field mapping,

an eye tracking system that generates eye-tracking data in response to aposition of a user's pupil,

a jitter sensor that generates jitter data, in response to movements;and

an augmentation stabilization control system that adjusts the positionof the light at the user's pupil, in response to the eye-tracking dataand the jitter data.

A63. A system for sensing, correcting and updating the location ofaugmentation imagery, in response to high frequency movements, thesystem comprising:

a controller that generates a field mapping, at a projector update rate,

a light generator that projects light associated with the field mapping,

an eye tracking system that generates eye-tracking data in response to aposition of a user's pupil,

a jitter sensor that generates jitter data, in response to movements;and

an augmentation stabilization control system that adjusts the positionof the light at the user's pupil, in response to the eye-tracking dataand the jitter data, and at an augmentation stabilization rate, which isat least as great as the projector update rate.

A64. A system for sensing, correcting and updating the location ofaugmentation imagery, in response to high frequency movements, thesystem comprising:

a controller that generates a field mapping,

a light generator that projects light associated with the field mapping,at a refresh rate,

an eye tracking system that generates eye-tracking data in response to aposition of a user's pupil,

a jitter sensor that generates jitter data, in response to movements;and

an augmentation stabilization control system that adjusts the positionof the light at the user's pupil, in response to the eye-tracking dataand the jitter data, and at an augmentation stabilization rate, which isat least as great as the refresh rate.

A65. A system according to any one of embodiments A59-A64, in which thejitter data is related to movements of the user's head.

A66. A system according to any one of embodiments A59-A64, in which thejitter data is related to movements of the user's eye.

A67. A system according to any one of embodiments A59-A64, in which thejitter data is related to movements of the light.

A68. A system according to any one of embodiments A59-A64, in which theprojector update rate is at least 30 Hz.

A69. A system according to any one of embodiments A59-A64, in which therefresh rate is at least 60 Hz.

A70. A system for generating render data, the system comprising:

render data,

an eye tracking system that generates eye-tracking data in response to aposition of a user's pupil; and

a controller that generates a field mapping in response to the renderdata and the eye-tracking data.

A71. The system of embodiment A70, wherein: the eye tracking system isconfigured to generate eye-tracking data at an eye-tracking rate atleast 11 Hz.

A72. The system of embodiment A71, wherein the eye-tracking rate is nogreater than 240 Hz.

A73. A system for generating render data, the system comprising:

render data,

an eye tracking system that generates eye-tracking data in response to aposition of a user's pupil; and

a controller that generates a color mapping in response to the renderdata and the eye-tracking data.

A74. A system for generating render data, the system comprising:

render data,

an eye tracking system that generates eye-tracking data in response to aposition of a user's pupil; and

a controller that generates a color mapping in response to the renderdata and the eye-tracking data, wherein the color mapping comprises atleast two resolutions.

A75. A system for generating render data and projecting light associatedwith the render data, the system comprising:

render data,

an eye tracking system that generates eye-tracking data in response to aposition of a user's pupil and a pose of a user's head,

a controller that generates a color mapping in response to the renderdata and the eye-tracking data, wherein the color mapping comprises atleast two resolutions; and

a light generator that projects light associated with the color mapping.

A76. A system for generating render data and projecting light associatedwith the render data, the system comprising:

render data,

an eye tracking system that generates eye-tracking data in response to aposition of a user's pupil; and

a controller that generates a color mapping in response to the renderdata and the eye-tracking data, wherein the color mapping, comprising afirst resolution and a second resolution,

a light generator that projects light associated with the color mapping,such that the light comprises: a first group of light rays, associatedwith the first resolution; and a second group of light rays, associatedwith the second resolution.

A77. A system for generating render data and projecting light associatedwith the render data, the system comprising:

render data,

an eye tracking system that generates eye-tracking data in response to aposition of a user's pupil; and

a controller that generates a color mapping in response to the renderdata and the eye-tracking data, wherein the color mapping, comprising afirst resolution and a second resolution,

a light generator that projects light associated with the color mapping,such that the light comprises: a first group of light rays associatedwith the first resolution, which are projected incident a central fieldof view of a user; and a second group of light rays associated with thesecond resolution, which are projected incident a peripheral field ofview of a user.

A78. A system for selectively reflecting and focusing an image, thesystem comprising:

a light, comprising: a first group of light rays and a second group oflight rays,

an optical element to modify the light, such that the second group oflight rays undergo greater divergence than the first group of lightrays; and

a switchable reflector located incident the light, which is configuredto selectively reflect the light incident the optical element.

A79. A system for selectively reflecting an image, the systemcomprising:

a light, comprising: a first group of light rays and a second group oflight rays,

a switchable partial reflector located incident the light, theswitchable partial reflector configured to selectively partiallyreflect: a reflected portion of the light, and selectively partiallytransmit: a non-reflected portion of the light,

a switchable reflector being configured to selectively retro-reflect thefirst group of light rays from the non-reflected portion of the light;and

a reflector being configured to retro-reflect the second group of lightrays from the reflected portion of the light, such that the first groupof light rays and the second group of light rays are, at leastpartially, recombined at the switchable partial reflector.

A80. The system of embodiment A79, wherein the reflector furthercomprises a curvature, such that a vergence of the second group of lightrays is modified.

A81. A system for selectively reflecting an image, the systemcomprising:

a light, comprising: a first group of light rays and a second group oflight rays,

a switchable partial reflector located incident the light, theswitchable partial reflector configured to selectively partiallyreflect: a reflected portion of the light, and selectively partiallytransmit: a non-reflected portion of the light,

a switchable reflector being configured to selectively retro-reflect thesecond group of light rays from the non-reflected portion of the light;and

a reflector being configured to retro-reflect the first group of lightrays from the reflected portion of the light, such that the first groupof light rays and the second group of light rays are, at leastpartially, recombined at the switchable partial reflector.

A82. A system for selectively reflecting an image, the systemcomprising:

a light, comprising: a first group of light rays and a second group oflight rays,

a first switchable reflector and a second switchable reflector that arelocated incident the light; and

the first switchable reflector being selectively reflective to the firstgroup of light rays and the second switchable reflector beingselectively reflective to the second group of light rays.

A83. A system for selectively reflecting and focusing an image, thesystem comprising:

A system for selectively reflecting and focusing an image, the systemcomprising:

a light, comprising: a first group of light rays and a second group oflight rays,

a first switchable reflector and a second switchable reflector that arelocated incident the light,

the first switchable reflector being selectively reflective to the firstgroup of light rays and the second switchable reflector beingselectively reflective to the second group of light rays,

a first optical element to modify a vergence of the first group of rays;and

a second optical element to modify a vergence of the second group oflight rays.

A84. The system of embodiment A83, wherein: the second optical elementfurther comprises a diffractive optical element.

A85. The system of embodiment A83, wherein: the first optical elementfurther comprises an imaging lens.

A86. The system of embodiment A83, wherein: the first optical elementfurther comprises an adjustable focus optic that selectively modifies avergence the first group of light rays.

A87. The system of embodiment A83, wherein: the second optical elementfurther comprises an adjustable focus optic that selectively modifies avergence of the second group of light rays.

A88. A system according to any one of embodiments A76-A87, in which thefirst group of light rays subtend an angle no greater than 10 degrees.

A89. A system according to any one of embodiments A76-A87, in which thesecond group of light rays subtend an angle no less than 10 degrees.

A90. A method for selectively reflecting an image, the methodcomprising:

projecting light associated with an image; and

selectively steering the light, and selectively reflecting the light;such that, the light is imaged by a user's eye.

A91. The method of embodiment A90, wherein the light comprises visiblespectrum light.

A92. The method of embodiment A90, wherein selectively reflecting thelight may occur at a rate of at least 30 Hz.

A93. The method of embodiment A92, wherein the rate is no greater than120 Hz.

A94. The method of embodiment A90, wherein the light has a width atleast as great as 1 mm.

A95. The method of embodiment A90, wherein the light is imaged within acentral field of view of the user.

A96. The method of embodiment A90, wherein the light comprisesaugmentation imagery.

A97. The method of embodiment A90, wherein selectively steering thelight comprises: introducing a phase gradient by linearly varying amultitude of spatial light modulator elements.

A98. A method for selectively reflecting an image, the methodcomprising:

input coupling light into a substrate,

selectively steering the light with an adjustable gradient optic; and

selectively reflecting the light with a switchable reflector, such thatlight being reflected is output coupled from the substrate.

A99. A method for selectively reflecting an image, the methodcomprising:

input coupling light by way of a first surface of a substrate,

selectively steering the light with an adjustable gradient optic; and

selectively reflecting the light with a switchable reflector, such thatlight being reflected is output coupled by way of a second surface fromthe substrate.

A100. A method for selectively reflecting an image, the methodcomprising:

input coupling light into a first surface of a substrate,

selectively steering the light; and

selectively reflecting the light, such that light being reflected isoutput coupled by way of a second surface of the substrate, and lightnot being reflected is output coupled by way of a third surface from thesubstrate.

A101. A method for selectively reflecting an image, the methodcomprising:

input coupling light into a substrate,

selectively reflecting the light, such that light being reflected isoutput coupled from the substrate; and

selectively shaping a wave front of the light being reflected.

A102. A method for selectively reflecting an image, the methodcomprising:

input coupling light into a substrate,

selectively steering the light,

selectively reflecting the light with a switchable reflector, such thatlight being reflected is output coupled from the substrate; and

selectively shaping a wave front of the light being reflected.

A103. A method according to any one of embodiments A101 and A102, inwhich selectively shaping a wave front of the light comprises: radiallyvarying a multitude of spatial light modulator elements.

A104. A method for selectively reflecting a light from a plurality oflocations, the method comprising:

directing a light incident an array of switchable reflectors; and

selectively reflecting the light from one or more switchable reflectors.

A105. The method of embodiment A104, wherein the light comprises visiblespectrum light.

A106. The method of embodiment A104, wherein selectively reflecting thelight may occur at a rate of at least 30 Hz.

A107. The method of embodiment A106, wherein the rate is no greater than120 Hz.

A108. The method of embodiment A104, wherein the light has a width atleast as great as 1 mm.

A109. The method of embodiment A104, wherein the light is imaged by aneye of a user.

A110. The method of embodiment A109, wherein the light is imaged withina central field of view of the user.

A111. The method of embodiment A104, wherein the light comprisesaugmentation imagery.

A112. A method for selectively reflecting a light incident a user'spupil, the method comprising:

tracking the position of a user's pupil,

directing a light incident an array of switchable reflectors; and

selectively reflecting the light from one or more switchable reflectors,such that the light is imaged onto the user's pupil.

A113. A method for selectively reflecting a light from a plurality oflocations, the method comprising:

directing a light incident an array of switchable reflectors,

selectively reflecting the light from one or more switchable reflectors;and

combining the light with an external light from one or more externallight source.

A114. A method for selectively reflecting a light from a plurality oflocations, the method comprising:

directing a light incident an array of switchable reflectors,

selectively steering the light; and

selectively reflecting the light from one or more switchable reflectors.

A115. The method of embodiment A114, wherein selectively steering thelight comprises: introducing a phase gradient by linearly varying amultitude of spatial light modulator elements.

A116. A method for selectively reflecting a light from a plurality oflocations, the method comprising:

directing a light incident an array of switchable reflectors,

selectively reflecting the light from one or more switchable reflectors;and

selectively shaping a wave front of the light being reflected from theone or more switchable reflectors.

A117. The method of embodiment A116, wherein selectively shaping a wavefront of the light comprises: radially varying a multitude of spatiallight modulator elements.

A118. A method according to any one of embodiments A104-A117, in whichthe array is curved.

A119. A method for selectively reflecting an image, the methodcomprising:

projecting light associated with an image; and

selectively steering the light, and selectively reflecting the light;such that, the light is imaged by a user's eye at a specified depth.

A120. A method for projecting accommodated imagery, the methodcomprising:

generating a color mapping, and a depth mapping,

projecting light, in response to the color mapping; and

selectively shaping a wave front of the light, in response to the depthmapping.

A121. The method of embodiment A120, wherein selectively shaping a wavefront of the light comprises radially varying a multitude of spatiallight modulator elements.

A122. The method of embodiment A120, wherein the light at an image planecomprises augmentation imagery.

A123. A method for projecting accommodated imagery, the methodcomprising:

tracking a position of a user's eye,

generating a color mapping and a depth mapping, in response to theposition of the user's eye,

projecting light, in response to the color mapping; and

selectively shaping a wave front of the light, in response to the depthmapping.

A124. A method for projecting accommodated imagery, the methodcomprising:

tracking a position of a user's eye,

determine a gaze vector in response to the position of the user's eye,

generating a color mapping, in response to the position of the user'seye,

generating at least one depth value associated with a distance along thegaze vector,

projecting light, in response to the color mapping; and

selectively shaping a wave front of the light, in response to the atleast one depth value.

A125. The method of embodiment A124, further comprising:

generating a calculated depth value associated with a distance along thegaze vector; and

generating at least one associated depth being proximal the calculateddepth value.

A126. A method for projecting accommodated imagery, the methodcomprising:

sequentially generating a color mapping, and a depth mapping,

projecting light, in response to the color mapping; and

selectively shaping a wave front of the light, in response to the depthmapping.

A127. A method for projecting accommodated imagery, the methodcomprising:

tracking a position of a user's eye,

sequentially generating a color mapping and a depth mapping, in responseto the position of the user's eye,

projecting light, in response to the color mapping; and

selectively shaping a wave front of the light, in response to the depthmapping.

A128. A method for projecting accommodated imagery, the methodcomprising:

tracking a position of a user's eye,

determine a gaze vector in response to the position of the user's eye,

sequentially generating a color mapping, in response to the position ofthe user's eye,

sequentially generating at least one depth value associated with adistance along the gaze vector,

projecting light, in response to the color mapping; and

selectively shaping a wave front of the light, in response to the atleast one depth value.

A129. A method for projecting accommodated imagery, the methodcomprising:

sequentially generating a color mapping, at a projector update rate,

projecting light, at a refresh rate, in response to the color mapping,

sequentially generating a depth mapping; and

selectively shaping a wave front of the light, at an accommodationupdate rate, in response to the depth mapping.

A130. The method of embodiment A129, wherein the accommodation updaterate is at least 30 Hz.

A131. The method of embodiment A129, wherein the accommodation updaterate is at least as fast as the refresh rate.

A132. The method of embodiment A129, wherein the projector update rateis no greater than 240 Hz.

A133. A method for sensing, correcting and updating a location ofaugmentation imagery, in response to jitter, the method comprising:

projecting light incident a user's pupil,

sensing jitter; and

adjusting a position of the light at the user's pupil, in response tothe jitter.

A134. The method of embodiment A133, wherein the light comprisesaugmentation imagery.

A135. The method of embodiment A133, wherein sensing the jittercomprises an inertial measurement unit.

A136. The method of embodiment A133, wherein adjusting the position ofthe light, further comprises: selectively steering the light.

A137. The method of embodiment A136, wherein selectively steering thelight, further comprises: introducing a phase gradient by linearlyvarying a multitude of spatial light modulator elements.

A138. The method of embodiment A136, wherein selectively steering thelight further comprises: varying a reflection angle of a reflector.

A139. The method of embodiment A133, wherein sensing the jitter, furthercomprises: measuring inertial changes of a device collocated with auser's head.

A140. The method of embodiment A133, wherein sensing the jitter, furthercomprises: measuring a position of the light.

A141. A method for sensing a location of augmentation imagery, inresponse to jitter, the method comprising:

generating a field mapping, at a projector update rate,

projecting light associated with the field mapping at a refresh rate atleast as great as the projector update rate; and

sensing jitter at a jitter sensing rate, at least as great as theprojector update rate.

A142. A method for sensing a location of augmentation imagery, inresponse to jitter, the method comprising:

generating a field mapping, at a projector update rate,

projecting light associated with the field mapping at a refresh rate atleast as great as the projector update rate,

tracking a position of a user's pupil at an eye-tracking rate; and

sensing jitter at a jitter sensing rate, at least as great as theprojector update rate.

A143. A method for sensing, correcting and updating a location ofaugmentation imagery, in response to jitter, the method comprising:

generating a field mapping, at a projector update rate,

projecting light associated with the field mapping,

sensing jitter; and

adjusting a position of the light at a user's pupil, in response to thejitter, at an augmentation stabilization rate that is at least as greatas the projector update rate.

A144. A method for sensing, correcting and updating a location ofaugmentation imagery, in response to jitter, the method comprising:

generating a field mapping, at a projector update rate,

projecting light associated with the field mapping,

tracking a position of a user's pupil,

sensing jitter; and

adjusting a position of the light at the user's pupil, in response tothe position of the user's pupil and the jitter.

A145. A method for sensing, correcting and updating a location ofaugmentation imagery, in response to jitter, the method comprising:

generating a field mapping, at a projector update rate,

projecting light associated with the field mapping,

tracking a position of a user's pupil,

sensing jitter; and

adjusting a position of the light at the user's pupil, in response tothe position of the user's pupil and the jitter, at an augmentationstabilization rate that is at least as great as the projector updaterate.

A146. A method for sensing, correcting and updating a location ofaugmentation imagery, in response to changes of a position of a user'spupil and jitter, the method comprising:

generating a field mapping,

projecting light associated with the field mapping, at a refresh rate,

tracking a position of a user's pupil

sensing jitter; and

adjusting a position of the light at the user's pupil, in response tothe position of the user's pupil and the jitter, at an augmentationstabilization rate that is at least as great as the refresh rate.

A147. A method for sensing, correcting and updating a location ofaugmentation imagery, in response to changes of a position of a user'spupil and jitter, the method comprising:

generating a field mapping, at a projector update rate,

projecting light associate with the field mapping,

sensing jitter; and

adjusting a position of the light incident a user's pupil, in responseto the jitter, by:

-   -   modifying the light to form a first pupil plane,    -   selectively steering the light at the first pupil plane,    -   modifying the light to form a second pupil plane,    -   selectively steering the light at the second pupil plane; and    -   modifying the light to form a third pupil plane, at the user's        pupil.

A148. A method for sensing, correcting and updating a location ofaugmentation imagery, in response to jitter, the method comprising:

generating a field mapping, at a projector update rate,

projecting light associate with the field mapping,

sensing jitter; and

adjusting a position of the light incident a user's pupil, in responseto the jitter, by:

-   -   selectively steering the light in a first plane,    -   selectively steering the light in a second plane; and    -   modifying the light to form a third pupil plane, at the user's        pupil.

A149. A method according to any one of embodiments 142 and 146, in whichthe refresh rate is at least 60 Hz.

A150. A method according to any one of embodiments 142-148, in which theprojector update rate is at least 30 Hz.

A151. A method for generating augmentation imagery, the methodcomprising:

tracking a position of a user's pupil; and

generating a field mapping associated with augmentation imagery, inresponse to render data and the position of the user's pupil.

A152. A method for generating augmentation imagery, the methodcomprising:

tracking a position of a user's pupil; and

generating a color mapping associated with augmentation imagery, inresponse to render data and the position of the user's pupil.

A153. A method for generating augmentation imagery, the methodcomprising:

tracking a position of a user's pupil; and

generating a color mapping associated with augmentation imagery, inresponse to render data and the position of the user's pupil; whereinthe color mapping comprises a plurality of resolutions.

A154. A method for generating augmentation imagery, the methodcomprising:

tracking a position of a user's pupil; and

generating a field mapping associated with augmentation imagery, inresponse to render data and the position of the user's pupil; whereinthe field mapping comprises nonlinear mapping.

A155. A method for generating augmentation imagery, the methodcomprising:

tracking a position of a user's pupil,

generating a color mapping associated with augmentation imagery, inresponse to render data and the position of the user's pupil; whereinthe color mapping comprises a plurality of resolutions; and

projecting light associated with the color mapping.

A156. A method for generating augmentation imagery, the methodcomprising:

tracking a position of a user's pupil,

generating a color mapping associated with augmentation imagery, inresponse to render data and the position of the user's pupil; whereinthe color mapping comprises a plurality of resolutions; and

projecting light associated with the color mapping, wherein the lightcomprises: a first group of light rays associated with one or moreresolutions of the color mapping; and a second group of light raysassociated with one or more resolutions of the color mapping.

A157. A method for generating augmentation imagery, the methodcomprising:

tracking a position of a user's pupil,

generating a color mapping associated with augmentation imagery, inresponse to render data and the position of the user's pupil; whereinthe color mapping comprises a plurality of resolutions; and

projecting first group of light rays associated with one or moreresolutions of the color mapping incident a central field of view of auser; and

projecting second group of light rays associated with one or moreresolutions of the color mapping incident a peripheral field of view ofa user.

A158. A method according to any one of embodiments A156 and A157, inwhich the first group of light rays subtend an angle no greater than 10degrees.

A159. A method according to any one of embodiments A159 and A157, inwhich the second group of light rays subtend an angle at least 10degrees.

A160. A method for generating augmentation imagery, the methodcomprising:

tracking a position of a user's pupil,

generating a field mapping associated with augmentation imagery, inresponse to render data and the position of the user's pupil; whereinthe field mapping comprises nonlinear mapping; and projecting lightassociated with the color mapping.

A161. A method for selectively reflecting and focusing an image, themethod comprising:

projecting a light, comprising a first group of light rays and a secondgroup of light rays,

selectively reflecting the light; and

modifying the light, such that a divergence of the first group of lightrays is less than a divergence of the second group of light rays.

A162. A method for selectively reflecting an image, the methodcomprising:

projecting a light, comprising a first group of light rays and a secondlight rays,

selectively reflecting the first light rays from a first switchablereflector; and

selectively reflecting the second group of light rays from a secondswitchable reflector.

A163. A method for selectively reflecting an image incident a centraland peripheral field of view of a user, the method comprising:

projecting a light, comprising a first group of light rays and a secondgroup of light rays,

-   -   selectively reflecting the first group of light rays from a        first switchable reflector, such that the first group of light        rays are imaged at a central field of view of a user; and    -   selectively reflecting the second group of light rays from a        second switchable reflector, such that the second group of light        rays are imaged a peripheral field of view of the user.

A164. The method of embodiment A163, further comprising: modifying avergence of the second group of light rays.

A165. The method of embodiment A163, further comprising: modifying avergence of the first group of light rays.

A166. The method of embodiment A163, further comprising: selectivelyshaping a wave front of the first group of light rays.

A167. A method for selectively reflecting and focusing an image, themethod comprising:

projecting a light, comprising a first group of light rays and a secondgroup of light rays,

selectively reflecting the first group of light rays from a firstswitchable reflector,

modifying a vergence of the first group of light rays,

selectively reflecting the second group of light rays from a secondswitchable reflector; and

modifying a vergence of the second group of light rays.

A168. A method for selectively reflecting and focusing of an image, themethod comprising:

projecting a light, comprising a first group of light rays and a secondgroup of light rays,

selectively reflecting the first group of light rays from a firstswitchable reflector,

selectively reflecting the second group of light rays from a secondswitchable reflector; and

modifying a vergence of the second group of light rays, such that thesecond group of light rays diverge more than the first group of lightrays.

A169. A method for selectively reflecting and focusing of an image, themethod comprising:

projecting a light, comprising a first group of light rays and a secondgroup of light rays,

selectively reflecting the first group of light rays from a firstswitchable reflector,

selectively reflecting the second group of light rays from a secondswitchable reflector; and

selectively modifying a vergence of the first group of light rays.

A170. A method for selectively reflecting and focusing an image, themethod comprising:

projecting a light, comprising a first group of light rays and a secondgroup of light rays,

modifying the light, such that the second group of light rays divergemore than the first group of light rays; and

selectively reflecting the light.

A171. A method for selectively reflecting an image, the methodcomprising:

projecting a light, comprising a first group of light rays and a secondgroup of light rays,

selectively partially reflecting a reflected portion of the light,

selectively partially transmitting a non-reflected portion of the light,

selectively retro-reflecting the first group of light rays from thenon-reflected portion of light,

retro-reflecting the second group of light rays from the reflectedportion of the light; and

partially recombining the first group of light rays and the second groupof light rays.

A172. A method for selectively reflecting an image, the methodcomprising:

projecting a light, comprising a first group of light rays and a secondgroup of light rays,

selectively partially reflecting a reflected portion of the light,

selectively partially transmitting a non-reflected portion of the light,

selectively retro-reflecting the second group of light rays from thenon-reflected portion of light,

retro-reflecting the first group of light rays from the reflectedportion of the light; and

partially recombining the first group of light rays and the second groupof light rays. The following is another list of embodiments contemplatedby the present disclosure:

B1. An optical system for introducing wave front changes, the systemcomprising:

a liquid crystal cell; and

an electrode layer affecting the liquid crystal cell, the electrodelayer being divided into a plurality of electrodes separated by contourlines; wherein the contour lines are associated with a wave front modeof an orthonormal basis set, such that along an individual contour linethe wave front mode has a value that is within 30% of a constant wavefront value; the optical system being configured to produce a wave frontchange associated with said wave front mode, such that the wave frontchange is at least 60% that of an ideal wave front change.

B2. The system of embodiment B1, wherein the orthonormal basis setfurther comprises: a Zernike basis set.

B3. The system of embodiment B1, wherein the wave front change has amaximum optical path difference of at least 3 waves.

B4. The system of embodiment B1, wherein the wave front change is atleast 80% that of an ideal wave front change.

B5. The system of embodiment B1, wherein the electrode layer furthercomprises one or more transparent resistors configured to bridgeadjacent electrodes.

B6. The system of embodiment B1, further comprising a floating electrodelayer between the electrode layer and the liquid crystal cell, thefloating electrode layer comprising: a plurality of floating electrodesarranged such that: floating electrodes are located between contourlines of the electrode layer and the liquid crystal cell.

B7. The system of embodiment B1, wherein the wave front mode is Zernikemode Noll index number 5.

B8. The system of embodiment Bl, further comprising:

a controller for controlling an electrical potential of one or moreelectrodes.

B9. An optical system comprising:

a liquid crystal cell; and

an electrode layer affecting the liquid crystal cell, the electrodelayer being divided into a plurality of electrodes separated byequi-phase contour lines belonging to Zernike mode Noll index number 5;the optical system being configured to produce a wave front change thathas a maximum optical path difference of at least 3 waves and isassociated with the Zernike mode Noll index number 5, such that the wavefront change is at least 60% that of an ideal oblique astigmatism wavefront change.

B10. An optical system comprising:

a liquid crystal cell; and

an electrode layer affecting the liquid crystal cell, the electrodelayer being divided into a plurality of electrodes separated byequi-phase contour lines belonging to Zernike mode Noll index number 6;the optical system being configured to produce a wave front change thathas a maximum optical path difference of at least 3 waves and isassociated with Zernike mode Noll index number 6, such that the wavefront change is at least 60% that of an ideal vertical astigmatism wavefront change.

B11. A foveated optical system, the foveated optical system comprising:

one or more optics for imaging a wide field of view,

a wave front correction optic, comprising a plurality of optical regionsarranged in a pattern associated with a wave front mode of anorthonormal basis set: the plurality of optical regions being adapted tovary optical path differences in response to corresponding electricalsignals; and

a controller for controlling the electrical signals, such that a wavefront error is reduced over a selectable region of interest within thewide field of view.

B12. The system of embodiment B11, wherein the orthonormal basis setfurther comprises: a Zernike basis set.

B13. The system of embodiment B11, wherein the wave front error isreduced by 60%.

B14. The system of embodiment B11, wherein the wave front error isreduced by 80%.

B15. The system of embodiment B11, wherein the optical path differenceshave a maximum optical path difference equal to at least three waves.

B16. The system of embodiment B11, wherein the wide field of view is atleast 50° in at least one axis.

B17. The system of embodiment B11, wherein the selectable region ofinterest is at least 2° in at least one axis.

B18. The system of embodiment B11, wherein the wave front mode isZernike mode Noll index number 5.

B19. The system of embodiment B11, wherein the wave front mode isZernike mode Noll index number 6.

B20. A foveated optical system, the foveated optical system comprising:

one or more optics for imaging a wide field of view that is at least 50°in at least one axis,

a wave front correction optic, comprising a plurality of opticalregions, arranged in a pattern associated with a wave front mode of aZernike basis set: the plurality of optical regions being adapted tovary optical path differences in response to corresponding electricalsignals, wherein the optical path differences have a maximum opticalpath difference of at least three waves; and

a controller for controlling the electrical signals, such that a wavefront error is reduced by 60% over a selectable region of interest thatis at least 2° in at least one axis within the wide field of view.

B21. A method of reducing a wave front error within a selectable regionof interest within a wide field of view, the method comprising:

imaging a wide field of view; and

controlling a plurality of optical path differences over a plurality ofregions arranged in a pattern associated with a wave front mode of anorthonormal basis set, such that the wave front error is reduced over aselectable region of interest within the wide field of view.

B22. The method of embodiment B21, wherein the orthonormal basis setfurther comprises: a Zernike basis set.

B23. The method of embodiment B21, wherein the wave front error isreduced by 60%.

B24. The method of embodiment B21, wherein the wave front error isreduced by 80%.

B25. The method of embodiment B21, wherein the optical path differenceshave a maximum optical path difference of at least three waves.

B26. The method of embodiment B21, wherein the wide field of view is atleast 50° in at least one axis.

B27. The method of embodiment B21, wherein the selectable region ofinterest is at least 2° in at least one axis.

B28. The method of embodiment B21, wherein the wave front mode isZernike mode Noll index number 5.

B29. The method of embodiment B21, wherein the wave front mode isZernike mode Noll index number 6.

B28. A method of reducing a wave front error within a selectable regionof interest within a wide field of view, the method comprising:

imaging a wide field of view that is at least 50° in at least one axis;and

controlling a plurality of optical path differences having a maximumoptical path difference of at least 3 waves, over a plurality of regionsarranged in a pattern associated with a wave front mode of a Zernikebasis set, such that the wave front error is reduced by 60% over aselectable region of interest that is at least 2° in at least one axis,within the wide field of view.

Therefore, the spirit and scope of the appended claims should not belimited to the description of the preferred versions contained herein.

What is claimed is:
 1. A system for projecting light onto an eye, thesystem comprising: a display configured to project light; a beamcombiner; a first optical system disposed between the display and thebeam combiner along a first optical path; a second optical systemdisposed between the display and the beam combiner along a secondoptical path, wherein the second optical path is different from thefirst optical path; and a switchable reflector configured to selectivelyswitch between: a reflective state, in which light incident upon theswitchable reflector is reflected by the switchable reflector, and anon-reflective state, in which light incident upon the switchablereflector is transmitted via the switchable reflector, the switchablereflector disposed between the display and the first and second opticalsystems along the first and second optical paths to direct the lightalong the first optical path to the beam combiner when in the reflectivestate, such that the light is reflected off the beam combiner andprojected upon an eye from a first direction, and to direct the lightalong the second optical path to the beam combiner when in thenon-reflective state, such that the light is reflected off the beamcombiner and projected upon the eye from a second direction differentfrom the first direction.
 2. The system of claim 1, wherein the light isviewable by the eye in a first field of view when the light is projectedupon the eye from the first direction, and the light is viewable by theeye in a second field of view when the light is projected upon the eyefrom the second direction.
 3. The system of claim 2, wherein the firstfield of view is at least 30°.
 4. The system of claim 2, wherein thefirst optical system is configured to project the light over the firstfield of view, the second optical system is configured to project thelight over the second field of view, and the first field of viewoverlaps the second field of view by at least 10°.
 5. The system ofclaim 1, further comprising: an eye tracking system configured todetermine an orientation of the eye relative to the beam combiner; and acontroller configured to switch the switchable reflector between thereflective state and the non-reflective state based at least in part onthe orientation of the eye.
 6. The system of claim 1, wherein theswitchable reflector has a clear aperture having a width of at least 2mm.
 7. The system of claim 1, wherein at least one of the first opticalsystem and the second optical system comprises a foveated opticalsystem.
 8. The system of claim 7, wherein the foveated optical systemcomprises a liquid crystal wave front corrector.
 9. The system of claim1, wherein the switchable reflector comprises a liquid crystal mirror.10. The system of claim 1, wherein at least one of the first opticalsystem and the second optical system is configured to collimate thelight, and wherein the beam combiner is partially reflective.
 11. Thesystem of claim 1, wherein the display is configured to linearlypolarize the light.
 12. The system of claim 1, wherein at least aportion of the beam combiner is curved to collimate light that isreflected to the eye.
 13. The system of claim 1, wherein the firstoptical path is longer than the second optical path.
 14. The system ofclaim 1, further comprising a reflector disposed between the switchablereflector and the first optical system along the first optical path. 15.The system of claim 1, wherein the switchable reflector reflectssubstantially all light incident upon the switchable reflector in thereflective state.
 16. The system of claim 1, wherein the displaycomprises: a first sub-display that projects a first group of light rayshaving a first resolution; and a second sub-display that projects asecond group of light rays having a second resolution different than thefirst resolution; and the projected light comprises the first group oflight rays and the second group of light rays.
 17. A method ofprojecting an image onto an eye, the method comprising: projecting lightdefining an image, via a display, onto a switchable reflector; andselectively switching the switchable reflector between a reflectivestate and a non-reflective state: wherein when the switchable reflectoris in the reflective state, the light incident upon the switchablereflector is reflected by the switchable reflector and directed along afirst optical path to a beam combiner, and the directed light reflectsoff the beam combiner and is projected upon an eye from a firstdirection; and wherein when the switchable reflector is in thenon-reflective state, the light incident upon the switchable reflectoris transmitted via the switchable reflector and directed along a secondoptical path, different from the first optical path, to the beamcombiner, and the directed light reflects off the beam combiner and isprojected upon the eye from a second direction.
 18. The method of claim17, further comprising tracking an orientation of the eye relative tothe beam combiner, and wherein the selectively switching the switchablereflector is performed based at least in part on the orientation of theeye.
 19. The method of claim 17, wherein, when the switchable reflectoris in the non-reflective state, the light incident upon the switchablereflector is transmitted via a clear aperture defined in the switchablereflector, the clear aperture having a width of about 2 mm to about 10mm.
 20. The method of claim 17, wherein projecting the light defining animage onto the switchable reflector comprises projecting a foveatedimage.
 21. The method of claim 17, wherein the switchable reflectorcomprises a liquid crystal mirror.
 22. The method of claim 17, whereinthe first optical path is defined in part by a first optical systemconfigured to project the directed light over a first field of view, andthe second optical path is defined in part by a second optical systemconfigured to project the directed light over a second field of view,and the first field of view overlaps the second field of view by atleast 10°.
 23. The method of claim 22, further comprising at least oneof the first optical system and the second optical system collimatingthe light that defines the image, and wherein the beam combiner ispartially reflective.
 24. The method of claim 17, wherein the displaylinearly polarizes the light that defines the image.
 25. The method ofclaim 17, further comprising the beam combiner collimating the lightcomprising the image that is reflected to the eye.
 26. The method ofclaim 17, wherein the first optical path is longer than the secondoptical path.
 27. The method of claim 17, further comprising reflectinglight from a reflector disposed between the switchable reflector and thefirst optical system along the first optical path.
 28. The method ofclaim 17, wherein the switchable reflector reflects substantially alllight incident upon the switchable reflector in the reflective state.29. The method of claim 17, wherein at least one of the first opticalsystem and the second optical system comprise a liquid crystal wavefront corrector.
 30. The method of claim 17, wherein projecting thelight comprises: projecting a first group of light rays having a firstresolution; and projecting a second group of light rays having a secondresolution different for the first resolution; and the projected lightcomprises the first group of light rays and the second group of lightrays.